Agones – Agones Documentation

Docs: System Diagram

Thu, 18 Apr 2024 00:00:00 +0000

Agones Control Plane

The Agones Control Plane consists of 4 Deployments:

NAME                READY   UP-TO-DATE   AVAILABLE   AGE
agones-allocator    3/3     3            3           40d
agones-controller   2/2     2            2           40d
agones-extensions   2/2     2            2           40d
agones-ping         2/2     2            2           40d

`agones-allocator`

agones-allocator provides a gRPC/REST service that translates allocation requests into GameServerAllocations. See Allocator Service for more information.

`agones-controller`

agones-controller maintains various control loops for all Agones CRDs (GameServer, Fleet, etc.). A single leader-elected Pod of the Deployment is active at any given time (see High Availability).

`agones-extensions`

agones-extensions is the endpoint for:

Agones-installed Kubernetes webhooks, which handle defaulting and validation for Agones CRs,
and the GameServerAllocation APIService, which handles GameServer allocations (either from the Allocator Service or the Kubernetes API).

`agones-ping` (not pictured)

agones-ping is a simple ping service for latency testing from your game client - see Latency Testing.

Agones CRDs

See Create a Game Server, Create a Game Server for examples of Agones CRDs in action, or the API Reference for more detail.

All of the Agones CRDs are controlled and updated by agones-controller. GameServer is additionally updated by the SDK Sidecar and agones-extensions (moving a GameServer from Ready to Allocated

Game Server Pod

Also pictured is an example Pod owned by a GameServer. Game clients typically connect to the Dedicated Game Server directly, or via a proxy like Quilkin. The game server server interfaces with Agones using the Client SDK, which is a thin wrapper around the SDK gRPC protocol. The SDK connects to the SDK Sidecar (sdk-server) in the same Pod, which handles the SDK business logic for e.g. health checks, Counters and Lists, etc.

Docs: Feature Stages

Thu, 26 Sep 2019 01:20:41 +0000

Supported Versions

Agones versions are expressed as x.y.z, where x is the major version, y is the minor version, and z is the patch version following Semantic Versioning terminology.

Agones Features

A feature within Agones can be in Alpha, Beta or Stable stage.

Feature Gates

Alpha and Beta features can be enabled or disabled through the agones.featureGates configuration option that can be found in the Helm configuration documentation.

The current set of alpha and beta feature gates:

Feature Name	Gate	Default	Stage	Since
DisableResyncOnSDKServer	`DisableResyncOnSDKServer`	Enabled	`Beta`	1.40.0
Support Passthrough on GKE Autopilot	`AutopilotPassthroughPort`	Disabled	`Alpha`	1.40.0
CountsAndLists	`CountsAndLists`	Disabled	`Alpha`	1.37.0
Support for Extended Duration Pods on GKE Autopilot (1.28+ only)	`GKEAutopilotExtendedDurationPods`	Disabled	`Alpha`	1.37.0
GameServer player capacity filtering on GameServerAllocations	`PlayerAllocationFilter`	Disabled	`Alpha`	1.14.0
Player Tracking	`PlayerTracking`	Disabled	`Alpha`	1.6.0
Example Gate (not in use)	`Example`	Disabled	None	0.13.0

Note

If you aren’t sure if Feature Flags have been set correctly, have a look at the The Feature Flag I enabled/disabled isn’t working as expected troubleshooting section.

Description of Stages

Alpha

An Alpha feature means:

Disabled by default.
Might be buggy. Enabling the feature may expose bugs.
Support for this feature may be dropped at any time without notice.
The API may change in incompatible ways in a later software release without notice.
Recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support.

Note

Please do try Alpha features and give feedback on them. This is important to ensure less breaking changes through the Beta period.

Beta

A Beta feature means:

Enabled by default, but able to be disabled through a feature gate.
The feature is well tested. Enabling the feature is considered safe.
Support for the overall feature will not be dropped, though details may change.
The schema and/or semantics of objects may change in incompatible ways in a subsequent beta or stable releases. When this happens, we will provide instructions for migrating to the next version. This may require deleting, editing, and re-creating API objects. The editing process may require some thought. This may require downtime for applications that rely on the feature.
Recommended for only non-business-critical uses because of potential for incompatible changes in subsequent releases. If you have multiple clusters that can be upgraded independently, you may be able to relax this restriction.

Note

Note: Please do try Beta features and give feedback on them! After they exit beta, it may not be practical for us to make more changes.

Stable

A Stable feature means:

The feature is enabled and the corresponding feature gate no longer exists.
Stable versions of features will appear in released software for many subsequent versions.

Feature Stage Indicators

There are a variety of features with Agones, how can we determine what stage each feature is in?

Below are indicators for each type of functionality that can be used to determine the feature stage for a given aspect of Agones.

Custom Resource Definitions (CRDs)

This refers to Kubernetes resource for Agones, such as GameServer, Fleet and GameServerAllocation.

New CRDs

For new resources, the stage of the resource will be indicated by the apiVersion of the resource.

For example: apiVersion: "agones.dev/v1" is a stable resource, apiVersion: "agones.dev/v1beta1" is a beta stage resource, and apiVersion: "agones.dev/v1alpha1" is an alpha stage resource.

New CRD attributes

When alpha and beta attributes are added to an existing stable Agones CRD, we will follow the Kubernetes Adding Unstable Features to Stable Versions Guide to optimise on the least amount of breaking changes for users as attributes progress through feature stages.

alpha and beta attributes will be added to the existing CRD as optional and documented with their feature stage. Attempting to populate these alpha and beta attributes on an Agones CRD will return a validation error if their accompanying Feature Flag is not enabled.

alpha and beta attributes can be subject to change of name and structure, and will result in breaking changes before moving to a stable stage. These changes will be outlined in release notes and feature documentation.

Agones Game Server SDK

Any alpha or beta Game Server SDK functionality will be a subpackage of the sdk package. For example , functionality found in a sdk.alphav1 package should be considered at the alpha feature stage.

Only experimental functionality will be found in any alpha and beta SDK packages, and as such may change as development occurs.

As SDK features move to through feature stages towards stable, the previous version of the SDK API will remain for at least one release to enable easy migration to the more stable feature stage (i.e. from alpha -> beta, beta -> stable)

Any other SDK functionality not marked as alpha or beta is assumed to be stable.

REST & gRPC APIs

REST and gRPC API will have versioned paths where appropriate to indicate their feature stage.

For example, a REST API with a prefix of v1alpha1 is an alpha stage feature: http://api.example.com/v1alpha1/exampleaction.

Similar to the SDK, any alpha or beta gRPC functionality will be a subpackage of the main API package. For example, functionality found in a api.alphav1 package should be considered at the alpha feature stage.

Docs: Matchmaker requests a GameServer from a Fleet

Tue, 27 Jul 2021 00:00:00 +0000

Sample `GameServerAllocation`

Since Agones will automatically add the label agones.dev/fleet to a GameServer of a given Fleet, we can use that label selector to target a specific Fleet by name. In this instance, we are targeting the Fleet xonotic.

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        agones.dev/fleet: xonotic

Next Steps:

Read the various specification references.
- Review the GameServer spec.
- Review the Fleet spec.
- Review the GameServerAllocation spec.
Review the specifics of Health Checking.
See all the commands the Client SDK provides - we only show a few here!
Check out the Allocator Service as a richer alternative to GameServerAllocation.
If you aren’t familiar with the term Pod, this should provide a reference.

Docs: Unreal Engine Game Server Client Plugin

Thu, 13 Jun 2019 10:17:50 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GameServer	✔️
Configuration	Watch	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	❌
Counters	SetCounterCount	❌
Counters	IncrementCounter	❌
Counters	DecrementCounter	❌
Counters	SetCounterCapacity	❌
Counters	GetCounterCapacity	❌
Lists	AppendListValue	❌
Lists	DeleteListValue	❌
Lists	SetListCapacity	❌
Lists	GetListCapacity	❌
Lists	ListContains	❌
Lists	GetListLength	❌
Lists	GetListValues	❌
Player Tracking	GetConnectedPlayers	✔️
Player Tracking	GetPlayerCapacity	✔️
Player Tracking	GetPlayerCount	✔️
Player Tracking	IsPlayerConnected	✔️
Player Tracking	PlayerConnect	✔️
Player Tracking	PlayerDisconnect	✔️
Player Tracking	SetPlayerCapacity	✔️

Additional methods have been added for ease of use (both of which are enabled by default):

Connect
- will call /gameserver till a succesful response is returned and then call /ready.
- disabled by setting bDisableAutoConnect to true.
- An event is broadcast with the GameServer data once the /gameserver call succeeds.
Health
- calls /health endpoint on supplied rate
- enabled by default with 10 second rate
- disabled by default by setting HealthRateSeconds to 0.

Both of the above are automatically kicked off in the BeginPlay of the component.

Download

Download the source from the Releases Page or directly from GitHub .

Resources

Unreal is a game engine that is used by anyone from hobbyists all the way through to huge AAA Game Studios.

With this in mind there is a vast amount to learn to run a production game using Unreal, even before you get to learning how it integrates with Agones. If you want to kick the tires with a starter project you will probably be fine with one of the starter projects out of the box.

However, as your Unreal/Agones project gets more advanced you will want to understand more about the engine itself and how it can be used to integrate with this project. There will be different ways of interacting via in Play In Editor ( PIE) versus running as an actual dedicated game server packaged into a container.

There are few helpful links for latest Unreal Engine 5:

UE5 Documentation Site
UE5 Dedicated Servers
- useful guide to getting started with dedicated servers in Unreal
UE5 Game Mode and Game State
UE5 Game Mode API Reference
- useful for hooking up calls to Agones
UE5 Game Session API Reference
- as above there are hooks in Game Session that can be used to call into Agones
UE5 Building & Packaging Games
- only building out Unreal game servers / clients, will also need to package into a container

If you use Unreal Engine 4, There are few helpful links for it:

Getting Started

This is a SDK inspired by the REST API to the Agones sidecars that allows engineers to communicate with the sidecar from either C++ or Blueprints.

Getting the Code

Easiest way to get this code is to clone the repository and drop the entire plugin folder into your own Plugins folder. This runs the plugin as a Project plugin rather than an engine plugin.

We could however turn this into a marketplace plugin that can be retrived from the marketplace directly into the UE editor.

Using C++ (UE5/UE4)

Add Plugin (in your own .uproject file)

  "Plugins": [
    {
      "Enabled": true,
      "Name": "Agones"
    }
  ],

Add Plugin (in your own *.Build.cs)

PublicDependencyModuleNames.AddRange(
    new[]
    {
        "Agones",
    });

Add component in header

#include "AgonesComponent.h"

UPROPERTY(EditAnywhere, BlueprintReadWrite)
UAgonesComponent* AgonesSDK;

Initialize component in GameMode

#include "AgonesComponent.h"
#include "Classes.h"

ATestGameMode::ATestGameMode()
{
	AgonesSDK = CreateDefaultSubobject<UAgonesComponent>(TEXT("AgonesSDK"));
}

Use the Agones component to call PlayerReady

void APlatformGameSession::PostLogin(APlayerController* NewPlayer)
{
  // Empty brances are for callbacks on success and errror.
  AgonesSDK->PlayerConnect("netspeak-player", {}, {});
}

Using Blueprints (UE5)

Add Component to your Blueprint GameMode
This will automatically call /health every 10 seconds and once /gameserver calls are succesful it will call /ready.
Accessing other functionality of Agones can be done via adding a node in Blueprints.

Using Blueprints (UE4)

Add Component to your Blueprint GameMode
This will automatically call /health every 10 seconds and once /gameserver calls are succesful it will call /ready.
Accessing other functionality of Agones can be done via adding a node in Blueprints.

Configuration Options

A number of options can be altered via config files in Unreal these are supplied via Game configuration eg. DefaultGame.ini.

[/Script/Agones.AgonesComponent]
HttpPort=1337
HealthRateSeconds=5.0
bDisableAutoConnect=true

Unreal Hooks

Within the Unreal GameMode and GameSession exist a number of useful existing funtions that can be used to fit in with making calls out to Agones.

A few examples are:

RegisterServer to call SetLabel, SetPlayerCapacity
PostLogin to call PlayerConnect
NotifyLogout to call PlayerDisconnect

Docs: Third Party Videos and Presentations

Thu, 03 Jan 2019 08:37:35 +0000

Presentations

Large-Scale Multiplayer Gaming on Kubernetes (Cloud Next ‘19)

Agones: Scaling Multiplayer Game Servers with Open Source (GDC 2019)

Intro to Agones: Scaling Multiplayer Game Servers with Kubernetes (Kubecon, 2018)

Google Cloud Next ‘18 London: Carl Dionne, Development Director at Ubisoft Montreal

Agones: Scaling Multiplayer Dedicated Game Servers with Kubernetes (Cloud Next ‘18)

Screencasts

Agones: How Do I Docker and Kubernetes? (Part 1 & Part 2)

Docs: Scheduling and Autoscaling

Thu, 03 Jan 2019 05:45:05 +0000

Cluster Autoscaler

Kubernetes has a cluster node autoscaler that works with a wide variety of cloud providers.

The default scheduling strategy (Packed) is designed to work with the Kubernetes autoscaler out of the box.

The autoscaler will automatically add Nodes to the cluster when GameServers don’t have room to be scheduled on the clusters, and then scale down when there are empty Nodes with no GameServers running on them.

This means that scaling Fleets up and down can be used to control the size of the cluster, as the cluster autoscaler will adjust the size of the cluster to match the resource needs of one or more Fleets running on it.

To enable and configure autoscaling on your cloud provider, check their connector implementation, or their cloud specific documentation.

Google Kubernetes Engine

Amazon Elastic Kubernetes Service

Cluster Autoscaler for EKS

Azure Kubernetes Service

Cluster Autoscaler on Azure Kubernetes Service (AKS) - Preview

Fleet Autoscaling

Fleet autoscaling is the only type of autoscaling that exists in Agones. It is currently available as a buffer autoscaling strategy or as a webhook driven strategy, such that you can provide your own autoscaling logic.

Have a look at the Create a Fleet Autoscaler quickstart, the Create a Webhook Fleet Autoscaler quickstart, and the Fleet Autoscaler Specification for details.

Autoscaling Concepts

To facilitate autoscaling, we need to combine several concepts and functionality, as described below.

Allocation Scheduling

Allocation scheduling refers to the order in which GameServers, and specifically their backing Pods are chosen from across the Kubernetes cluster within a given Fleet when allocation occurs.

Pod Scheduling

Each GameServer is backed by a Kubernetes Pod. Pod scheduling refers to the strategy that is in place that determines which node in the Kubernetes cluster the Pod is assigned to, when it is created.

Fleet Scale Down Strategy

Fleet Scale Down strategy refers to the order in which the GameServers that belong to a Fleet are deleted, when Fleets are shrunk in size.

Fleet Scheduling

There are two scheduling strategies for Fleets - each designed for different types of Kubernetes Environments.

Packed

apiVersion: "agones.dev/v1"
kind: Fleet
metadata:
  name: simple-game-server
spec:
  replicas: 100
  scheduling: Packed
  template:
    spec:
      ports:
      - containerPort: 7654
      template:
        spec:
          containers:
          - name: simple-game-server
            image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32

This is the default Fleet scheduling strategy. It is designed for dynamic Kubernetes environments, wherein you wish to scale up and down as load increases or decreases, such as in a Cloud environment where you are paying for the infrastructure you use.

It attempts to pack as much as possible into the smallest set of nodes, to make scaling infrastructure down as easy as possible.

This affects the Cluster autoscaler, Allocation Scheduling, Pod Scheduling and Fleet Scale Down Scheduling.

Cluster Autoscaler

When using the “Packed” strategy, Agones will ensure that the Cluster Autoscaler doesn’t attempt to evict and move GameServer Pods onto new Nodes during gameplay.

If a gameserver can tolerate being evicted (generally in combination with setting an appropriate graceful termination period on the gameserver pod) and you want the Cluster Autoscaler to compact your cluster by evicting game servers when it would allow the Cluster Autoscaler to reduce the number of nodes in the cluster, Controlling Disruption describes how to choose the .eviction setting appropriate for your GameServer or Fleet.

Allocation Scheduling Strategy

Under the “Packed” strategy, allocation will prioritise allocating GameServers to nodes that are running on Nodes that already have allocated GameServers running on them.

Pod Scheduling Strategy

Under the “Packed” strategy, Pods will be scheduled using the PodAffinity with a preferredDuringSchedulingIgnoredDuringExecution affinity with hostname topology. This attempts to group together GameServer Pods within as few nodes in the cluster as it can.

Note

The default Kubernetes scheduler doesn’t do a perfect job of packing, but it’s a good enough job for what we need - at least at this stage.

Fleet Scale Down Strategy

With the “Packed” strategy, Fleets will remove Ready GameServers from Nodes with the least number of Ready and Allocated GameServers on them. Attempting to empty Nodes so that they can be safely removed.

Distributed

apiVersion: "agones.dev/v1"
kind: Fleet
metadata:
  name: simple-game-server
spec:
  replicas: 100
  scheduling: Distributed
  template:
    spec:
      ports:
      - containerPort: 7654
      template:
        spec:
          containers:
          - name: simple-game-server
            image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32

This Fleet scheduling strategy is designed for static Kubernetes environments, such as when you are running Kubernetes on bare metal, and the cluster size rarely changes, if at all.

This attempts to distribute the load across the entire cluster as much as possible, to take advantage of the static size of the cluster.

Note

Distributed scheduling does not set a PodAffinity on GameServer Pods, and instead assumes that the default scheduler for your cluster will distribute the GameServer Pods across the cluster by default.

If your default scheduler does not do this, you may wish to set your own PodAffinity to spread the load across the cluster, or update the default scheduler to provide this functionality.

This affects Allocation Scheduling, Pod Scheduling and Fleet Scale Down Scheduling.

Cluster Autoscaler

Since this strategy is not aimed at clusters that autoscale, this strategy does nothing for the cluster autoscaler.

Allocation Scheduling Strategy

Under the “Distributed” strategy, allocation will prioritise allocating GameServers to nodes that have the least number of allocated GameServers on them.

Pod Scheduling Strategy

Under the “Distributed” strategy, Pod scheduling is provided by the default Kubernetes scheduler, which will attempt to distribute the GameServer Pods across as many nodes as possible.

Fleet Scale Down Strategy

With the “Distributed” strategy, Fleets will remove Ready GameServers from Nodes with at random, to ensure a distributed load is maintained.

Docs: GameServer Specification

Thu, 03 Jan 2019 03:58:48 +0000

A full GameServer specification is available below and in the example folder for reference :

apiVersion: "agones.dev/v1"
kind: GameServer
# GameServer Metadata
# https://v1-28.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#objectmeta-v1-meta
metadata:
  # generateName: "gds-example" # generate a unique name, with the given prefix
  name: "gds-example" # set a fixed name
spec:
  # if there is more than one container, specify which one is the game server
  container: example-server
  # Array of ports that can be exposed as direct connections to the game server container
  ports:
    # name is a descriptive name for the port
  - name: default
    # portPolicy has three options:
    # - "Dynamic" (default) the system allocates a free hostPort for the gameserver, for game clients to connect to
    # - "Static", user defines the hostPort that the game client will connect to. Then onus is on the user to ensure that the
    # port is available. When static is the policy specified, `hostPort` is required to be populated
    # - "Passthrough" dynamically sets the `containerPort` to the same value as the dynamically selected hostPort.
    #      This will mean that users will need to lookup what port has been opened through the server side SDK.
    portPolicy: Static
    # The name of the container to open the port on. Defaults to the game server container if omitted or empty.
    container: simple-game-server
    # the port that is being opened on the game server process
    containerPort: 7654
    # the port exposed on the host, only required when `portPolicy` is "Static". Overwritten when portPolicy is "Dynamic".
    hostPort: 7777
    # protocol being used. Defaults to UDP. TCP and TCPUDP are other options
    # - "UDP" (default) use the UDP protocol
    # - "TCP", use the TCP protocol
    # - "TCPUDP", uses both TCP and UDP, and exposes the same hostPort for both protocols.
    #       This will mean that it adds an extra port, and the first port is set to TCP, and second port set to UDP
    protocol: UDP
  # Health checking for the running game server
  health:
    # Disable health checking. defaults to false, but can be set to true
    disabled: false
    # Number of seconds after the container has started before health check is initiated. Defaults to 5 seconds
    initialDelaySeconds: 5
    # If the `Health()` function doesn't get called at least once every period (seconds), then
    # the game server is not healthy. Defaults to 5
    periodSeconds: 5
    # Minimum consecutive failures for the health probe to be considered failed after having succeeded.
    # Defaults to 3. Minimum value is 1
    failureThreshold: 3
  # Parameters for game server sidecar
  sdkServer:
    # sdkServer log level parameter has three options:
    #  - "Info" (default) The SDK server will output all messages except for debug messages
    #  - "Debug" The SDK server will output all messages including debug messages
    #  - "Error" The SDK server will only output error messages
    logLevel: Info
    # grpcPort and httpPort control what ports the sdkserver listens on.
    # Starting with Agones 1.2 the default grpcPort is 9357 and the default
    # httpPort is 9358. In earlier releases, the defaults were 59357 and 59358
    # respectively but as these were in the ephemeral port range they could
    # conflict with other TCP connections.
    grpcPort: 9357
    httpPort: 9358
  # [Stage:Alpha]
  # [FeatureFlag:PlayerTracking]
  # Players provides the configuration for player tracking features.
  # Commented out since Alpha, and disabled by default
  # players:
  #   # set this GameServer's initial player capacity
  #   initialCapacity: 10
  #
  # [Stage:Alpha]
  # [FeatureFlag:CountsAndLists]
  # Counts and Lists provides the configuration for generic (player, room, session, etc.) tracking features.
  # Commented out since Alpha, and disabled by default
  # counters: # counters are int64 counters that can be incremented and decremented by set amounts. Keys must be declared at GameServer creation time.
  #   rooms: # arbitrary key.
  #     count: 1 # initial value can be set.
  #     capacity: 100 # (Optional) Defaults to 1000 and setting capacity to max(int64) may lead to issues and is not recommended. See GitHub issue https://github.com/googleforgames/agones/issues/3636 for more details.
  # lists: # lists are lists of values stored against this GameServer that can be added and deleted from. Keys must be declared at GameServer creation time.
  #   players: # an empty list, with a capacity set to 10.
  #     capacity: 10 # capacity value, defaults to 1000.
  #   rooms: # note that it is allowed to have the same key name with one used in counters
  #     capacity: 333
  #     values: # initial values can also be set for lists
  #       - room1
  #       - room2
  #       - room3
  #  
  # Pod template configuration
  # https://v1-28.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podtemplate-v1-core
  template:
    # pod metadata. Name & Namespace is overwritten
    metadata:
      labels:
        myspeciallabel: myspecialvalue
    # Pod Specification
    spec:
      containers:
      - name: simple-game-server
        image:  us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32
        imagePullPolicy: Always
      # nodeSelector is a label that can be used to tell Kubernetes which host
      # OS to use. For Windows game servers uncomment the nodeSelector
      # definition below.
      # Details: https://kubernetes.io/docs/setup/production-environment/windows/user-guide-windows-containers/#ensuring-os-specific-workloads-land-on-the-appropriate-container-host
      # nodeSelector:
      #   kubernetes.io/os: windows

Since Agones defines a new Custom Resources Definition (CRD) we can define a new resource using the kind GameServer with the custom group agones.dev and API version v1.

You can use the metadata field to target a specific namespaces but also attach specific annotations and labels to your resource. This is a very common pattern in the Kubernetes ecosystem.

The length of the name field of the Gameserver should not exceed 63 characters.

The spec field is the actual GameServer specification and it is composed as follow:

container is the name of container running the GameServer in case you have more than one container defined in the pod. If you do, this is a mandatory field. For instance this is useful if you want to run a sidecar to ship logs.
ports are an array of ports that can be exposed as direct connections to the game server container
- name is an optional descriptive name for a port
- portPolicy has three options:
  - Dynamic (default) the system allocates a random free hostPort for the gameserver, for game clients to connect to.
  - Static, user defines the hostPort that the game client will connect to. Then onus is on the user to ensure that the port is available. When static is the policy specified, hostPort is required to be populated.
  - Passthrough dynamically sets the containerPort to the same value as the dynamically selected hostPort. This will mean that users will need to lookup what port to open through the server side SDK before starting communications.
- container (Alpha) the name of the container to open the port on. Defaults to the game server container if omitted or empty.
- containerPort the port that is being opened on the game server process, this is a required field for Dynamic and Static port policies, and should not be included in Passthrough configuration.
- protocol the protocol being used. Defaults to UDP. TCP and TCPUDP are other options.
health to track the overall healthy state of the GameServer, more information available in the health check documentation.
sdkServer defines parameters for the game server sidecar
- logging field defines log level for SDK server. Defaults to “Info”. It has three options:
  - “Info” (default) The SDK server will output all messages except for debug messages
  - “Debug” The SDK server will output all messages including debug messages
  - “Error” The SDK server will only output error messages
- grpcPort the port that the SDK Server binds to for gRPC connections
- httpPort the port that the SDK Server binds to for HTTP gRPC gateway connections
players (Alpha, behind “PlayerTracking” feature gate), sets this GameServer’s initial player capacity
counters (Alpha, requires “CountsAndLists” feature flag) are int64 counters with a default capacity of 1000 that can be incremented and decremented by set amounts. Keys must be declared at GameServer creation time. Note that setting the capacity to max(int64) may lead to issues.
lists (Alpha, requires “CountsAndLists” feature flag) are lists of values stored against this GameServer that can be added and deleted from. Key must be declared at GameServer creation time.
template the pod spec template to run your GameServer containers, see for more information.

Note

The GameServer resource does not support updates. If you need to make regular updates to the GameServer spec, consider using a <a href="/site/docs/reference/fleet/">Fleet</a>.

Stable Network ID

If you want to connect to a GameServer from within your Kubernetes cluster via a convention based DNS entry, each Pod attached to a GameServer automatically derives its hostname from the name of the GameServer.

To create internal DNS entries within the cluster, a group of Pods attached to GameServers can use a Headless Service to control the domain of the Pods, along with providing a subdomain value to the GameServer PodTemplateSpec to provide all the required details such that Kubernetes will create a DNS record for each Pod behind the Service.

You are also responsible for setting the labels on the GameServer.Spec.Template.Metadata to set the labels on the created Pods and creating the Headless Service responsible for the network identity of the pods, Agones will not do this for you, as a stable DNS record is not required for all use cases.

To ensure that the hostName value matches RFC 1123, any . values in the GameServer name are replaced by - when setting the underlying Pod.Spec.HostName value.

GameServer State Diagram

The following diagram shows the lifecycle of a GameServer.

Game Servers are created through Kubernetes API (either directly or through a Fleet) and their state transitions are orchestrated by:

GameServer controller, which allocates ports, launches Pods backing game servers and manages their lifetime
Allocation controller, which marks game servers as Allocated to handle a game session
SDK, which manages health checking and shutdown of a game server session

Primary Address vs Addresses

GameServer.Status has two fields which reflect the network address of the GameServer: address and addresses. The address field is a policy-based choice of “primary address” that will work for many use cases, and will always be one of the addresses. The addresses field contains every address in the Node.Status.addresses and Pod.Status.podIPs (to allow a direct pod access), representing all known ways to reach the GameServer over the network.

To choose address from addresses, Agones looks for the following address types, in highest to lowest priorty:

ExternalDNS
ExternalIP
InternalDNS
InternalIP

e.g. if any ExternalDNS address is found in the respective Node, it is used as the address. (PodIP is not considered for address.)

The policy for address will work for many use-cases, but for some advanced cases, such as IPv6 enablement, you may need to evaluate all addresses and pick the addresses that best suits your needs.

Docs: Quickstart: Create a Game Server

Wed, 02 Jan 2019 06:35:31 +0000

Prerequisites

The following prerequisites are required to create a GameServer:

A Kubernetes cluster with the UDP port range 7000-8000 open on each node. Creating the firewall.
Agones controller installed in the targeted cluster
kubectl properly configured
Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Agones.

For the purpose of this guide we’re going to use the simple-game-server example as the GameServer container. This example is a very simple UDP server written in Go. Don’t hesitate to look at the code of this example for more information.

Objectives

Create a GameServer in Kubernetes using Agones custom resource.
Get information about the GameServer such as IP address, port and state.
Connect to the GameServer.

1. Create a GameServer

Let’s create a GameServer using the following command :

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserver.yaml

You should see a successful output similar to this :

gameserver.agones.dev/simple-game-server-4ss4j created

This has created a GameServer record inside Kubernetes, which has also created a backing Pod to run our simple udp game server code in. If you want to see all your running GameServers you can run:

kubectl get gameservers

It should look something like this:

NAME                       STATE     ADDRESS       PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   3m

You can also see the Pod that got created by running kubectl get pods, the Pod will be prefixed by simple-game-server.

NAME                        READY     STATUS    RESTARTS   AGE
simple-game-server-7pjrq    2/2       Running   0          5m

As you can see above it says READY: 2/2 this means there are two containers running in this Pod, this is because Agones injected the SDK sidecar for readiness and health checking of your Game Server.

For the full details of the YAML file head to the GameServer Specification Guide

2. Fetch the GameServer Status

Let’s wait for the GameServer state to become Ready. You can use the watch tool to see the state change. If your operating system does not have watch, manually run kubectl describe gameserver until the state changes.

watch kubectl describe gameserver

Name:         simple-game-server-7pjrq
Namespace:    default
Labels:       <none>
Annotations:  agones.dev/sdk-version: 0.9.0-764fa53
API Version:  agones.dev/v1
Kind:         GameServer
Metadata:
  Creation Timestamp:  2019-02-27T15:06:20Z
  Finalizers:
    agones.dev
  Generate Name:     simple-game-server-
  Generation:        1
  Resource Version:  30377
  Self Link:         /apis/agones.dev/v1/namespaces/default/gameservers/simple-game-server-7pjrq
  UID:               3d7ac3e1-3aa1-11e9-a4f5-42010a8a0019
Spec:
  Container:  simple-game-server
  Health:
    Failure Threshold:      3
    Initial Delay Seconds:  5
    Period Seconds:         5
  Ports:
    Container Port:  7654
    Host Port:       7190
    Name:            default
    Port Policy:     Dynamic
    Protocol:        UDP
  Scheduling:        Packed
  Template:
    Metadata:
      Creation Timestamp:  <nil>
    Spec:
      Containers:
        Image:  us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32
        Name:   simple-game-server
        Resources:
          Limits:
            Cpu:     20m
            Memory:  32Mi
          Requests:
            Cpu:     20m
            Memory:  32Mi
Status:
  Address:    35.233.183.43
  Node Name:  agones
  Ports:
    Name:  default
    Port:  7190
  State:   Ready
Events:
  Type    Reason          Age   From                   Message
  ----    ------          ----  ----                   -------
  Normal  PortAllocation  34s   gameserver-controller  Port allocated
  Normal  Creating        34s   gameserver-controller  Pod simple-game-server-7pjrq created
  Normal  Scheduled       34s   gameserver-controller  Address and port populated
  Normal  Ready           27s   gameserver-controller  SDK.Ready() executed

If you look towards the bottom, you can see there is a Status > State value. We are waiting for it to move to Ready, which means that the game server is ready to accept connections.

You might also be interested to see the Events section, which outlines when various lifecycle events of the GameServer occur. We can also see when the GameServer is ready on the event stream as well - at which time the Status > Address and Status > Ports > Port have also been populated, letting us know what IP and port our client can now connect to!

Let’s retrieve the IP address and the allocated port of your Game Server :

kubectl get gs

This should output your Game Server IP address and ports, eg:

NAME                       STATE   ADDRESS         PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   4m

Note

If you have Agones installed on minikube, or other local Kubernetes tooling, and you are having issues connecting to the GameServer, please check the Minikube local connection workarounds.

3. Connect to the GameServer

You can now communicate with the Game Server, by running:

nc -u {IP} {PORT}

Now write any text you would like, and hit <Enter>. You should see your text echoed back, like so:

nc -u 35.233.183.43 7190
Hello World !
ACK: Hello World !

You can finally type EXIT and hit <Enter>, which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

To exit nc you can press <ctrl>+c.

If you run kubectl describe gameserver again - either the GameServer will be gone completely, or it will be in Shutdown state, on the way to being deleted.

Note

If you do not have netcat installed (i.e. you get a response of nc: command not found), you can install netcat by running sudo apt install netcat.

If you are on Windows, you can alternatively install netcat on WSL, or download a version of netcat for Windows from nmap.org.

Next Step

If you want to use your own GameServer container make sure you have properly integrated the Agones SDK.

Docs: Google Kubernetes Engine

Mon, 01 Jan 0001 00:00:00 +0000

Before you begin

Take the following steps to enable the Kubernetes Engine API:

Visit the Kubernetes Engine page in the Google Cloud Platform Console.
Create or select a project.
Wait for the API and related services to be enabled. This can take several minutes.
Enable billing for your project.

If you are not an existing GCP user, you may be able to enroll for a $300 US Free Trial credit.

Choosing a shell

To complete this quickstart, we can use either Google Cloud Shell or a local shell.

Google Cloud Shell is a shell environment for managing resources hosted on Google Cloud Platform (GCP). Cloud Shell comes preinstalled with the gcloud and kubectl command-line tools. gcloud provides the primary command-line interface for GCP, and kubectl provides the command-line interface for running commands against Kubernetes clusters.

If you prefer using your local shell, you must install the gcloud and kubectl command-line tools in your environment.

Cloud shell

To launch Cloud Shell, perform the following steps:

Go to Google Cloud Platform Console
From the top-right corner of the console, click the Activate Google Cloud Shell button:
A Cloud Shell session opens inside a frame at the bottom of the console. Use this shell to run gcloud and kubectl commands.

Local shell

To install gcloud and kubectl, perform the following steps:

Install the Google Cloud SDK, which includes the gcloud command-line tool.
Initialize some default configuration by running the following command.
- When asked Do you want to configure a default Compute Region and Zone? (Y/n)?, enter Y and choose a zone in your geographical region of choice.
```
gcloud init
```
Install the kubectl command-line tool by running the following command:
```
gcloud components install kubectl
```

Choosing a Regional or Zonal Cluster

You will need to pick a geographical region or zone where you want to deploy your cluster, and whether to create a regional or zonal cluster. We recommend using a Regional cluster, as the zonal GKE control plane can go down temporarily to adjust for cluster resizing, automatic upgrades and repairs.

After choosing a cluster type, choose a region or zone. The region you chose is COMPUTE_REGION below. (Note that if you chose a zone, replace --region=[COMPUTE_REGION] with --zone=[COMPUTE_ZONE] in commands below.)

Choosing a Release Channel and Optional Version

We recommend using the regular release channel, which offers a balance between stability and freshness. If you’d like to read more, see our guide on Release Channels. The release channel you chose is RELEASE_CHANNEL below.

(Optional) During cluster creation, to set a specific available version in the release channel, use the --cluster-version=[VERSION] flag, e.g. --cluster-version=1.28. Be sure to choose a version supported by Agones. (If you rely on release channels, the latest Agones release should be supported by the default versions of all channels.)

Choosing a GKE cluster mode

A cluster consists of at least one control plane machine and multiple worker machines called nodes. In Google Kubernetes Engine, nodes are Compute Engine virtual machine instances that run the Kubernetes processes necessary to make them part of the cluster.

Agones supports both GKE Standard mode and GKE Autopilot mode. Agones GameServer and Fleet manifests that work on Standard are compatible on Autopilot with some constraints, described in the following section. We recommend running GKE Autopilot clusters, if you meet the constraints.

You can’t convert existing Standard clusters to Autopilot; create new Autopilot clusters instead.

Agones on GKE Autopilot

Autopilot is GKE’s fully-managed mode. GKE configures, maintains, scales, and upgrades nodes for you, which can reduce your maintenance and operating overhead. You only pay for the resources requested by your running Pods, and you don’t pay for unused node capacity or Kubernetes system workloads.

This section describes the Agones-specific considerations in Autopilot clusters. For a general comparison between Autopilot and Standard, refer to Choose a GKE mode of operation.

Autopilot nodes are, by default, optimized for most workloads. If some of your workloads have broad compute requirements such as Arm architecture or a minimum CPU platform, you can also choose a compute class that meets that requirement. However, if you have specialized hardware needs that require fine-grained control over machine configuration, consider using GKE Standard.

Agones on Autopilot has pre-configured opinionated constraints. Evaluate whether these constraints impact your workloads:

Operating system: No Windows containers.
Resource requests: Autopilot has pre-determined minimum Pod resource requests. If your game servers require less than those minimums, use GKE Standard.
Scheduling strategy: Packed is supported, which is the Agones default. Distributed is not supported.
Host port policy: Dynamic is supported, which is the Agones default. Static and Passthrough are not supported.
Seccomp profile: Agones sets the seccomp profile to Unconfined to avoid unexpected container creation delays that might occur because Autopilot enables the RuntimeDefault seccomp profile.
Pod disruption policy: eviction.safe: Never is supported, which is the Agones default. eviction.safe: Always is supported. eviction.safe: OnUpgrade is not supported. If your game sessions exceed one hour, refer to Considerations for long sessions.

Choosing a GCP network

By default, gcloud and the Cloud Console use the VPC named default for all new resources. If you plan to create a dual-stack IPv4/IPv6 cluster cluster, special considerations need to be made. Dual-stack clusters require a dual-stack subnet, which are only supported in custom mode VPC networks. For a new dual-stack cluster, you can either:

create a new custom mode VPC,
or if you wish to continue using the default network, you must switch it to custom mode. After switching a network to custom mode, you will need to manually manage subnets within the default VPC.

Once you have a custom mode VPC, you will need to choose whether to use an existing subnet or create a new one - read VPC-native guide on creating a dual-stack cluster, but don’t create the cluster just yet - we’ll create the cluster later in this guide. To use the network and/or subnetwork you just created, you’ll need to add --network and --subnetwork, and for GKE Standard, possibly --stack-type and --ipv6-access-type, depending on whether you created the subnet simultaneously with the cluster.

Creating the firewall

We need a firewall to allow UDP traffic to nodes tagged as game-server via ports 7000-8000. These firewall rules apply to cluster nodes you will create in the next section.

gcloud compute firewall-rules create gke-agones-game-server-firewall \
  --allow udp:7000-8000 \
  --target-tags game-server \
  --description "Firewall to allow game server udp traffic"

Creating the cluster

Create a GKE cluster in which you’ll install Agones. You can use GKE Standard mode or GKE Autopilot mode. You can read more about choosing a cluster mode above.

Create an Autopilot mode cluster for Agones

Choose a Release Channel (Autopilot clusters must be on a Release Channel).

Create the cluster:

gcloud container clusters create-auto [CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --release-channel=[RELEASE_CHANNEL] \
  --autoprovisioning-network-tags=game-server

Replace the following:

[CLUSTER_NAME]: The name of your cluster.
[COMPUTE_REGION]: the GCP region to create the cluster in.
[RELEASE_CHANNEL]: one of rapid, regular, or stable, chosen above. The default is regular.

Flag explanations:

--region: The compute region you chose above.
--release-channel: The release channel you chose above.
--autoprovisioning-network-tags: Defines the tags that will be attached to new nodes in the cluster. This is to grant access through ports via the firewall created above.

Create a Standard mode cluster for Agones

Create the cluster:

gcloud container clusters create [CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --release-channel=[RELEASE_CHANNEL] \
  --tags=game-server \
  --scopes=gke-default \
  --num-nodes=4 \
  --enable-image-streaming \
  --machine-type=e2-standard-4

Replace the following:

[CLUSTER_NAME]: The name of the cluster you want to create
[COMPUTE_REGION]: The GCP region to create the cluster in, chosen above
[RELEASE_CHANNEL]: The GKE release channel, chosen above

Flag explanations:

--region: The compute region you chose above.
--release-channel: The release channel you chose above.
--tags: Defines the tags that will be attached to new nodes in the cluster. This is to grant access through ports via the firewall created above.
--scopes: Defines the Oauth scopes required by the nodes.
--num-nodes: The number of nodes to be created in each of the cluster’s zones. Default: 4. Depending on the needs of your game, this parameter should be adjusted.
--enable-image-streaming: Use Image streaming to pull container images, which leads to significant improvements in initialization times. Limitations apply to enable this feature.
--machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.

(Optional) Creating a dedicated node pool

Create a dedicated node pool for the Agones resources to be installed in. If you skip this step, the Agones controllers will share the default node pool with your game servers, which is fine for experimentation but not recommended for a production deployment.

gcloud container node-pools create agones-system \
  --cluster=[CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --node-taints agones.dev/agones-system=true:NoExecute \
  --node-labels agones.dev/agones-system=true \
  --num-nodes=1 \
  --machine-type=e2-standard-4

Replace the following:

[CLUSTER_NAME]: The name of the cluster you created
[COMPUTE_REGION]: The GCP region to create the cluster in, chosen above

Flag explanations:

--cluster: The name of the cluster you created.
--region: The compute region you chose above.
--node-taints: The Kubernetes taints to automatically apply to nodes in this node pool.
--node-labels: The Kubernetes labels to automatically apply to nodes in this node pool.
--num-nodes: The number of nodes per cluster zone. For regional clusters, --num-nodes=1 creates one node in 3 separate zones in the region, giving you faster recovery time in the event of a node failure.
--machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.

(Optional) Creating a metrics node pool

Create a node pool for Metrics if you want to monitor the Agones system using Prometheus with Grafana or Cloud Logging and Monitoring.

gcloud container node-pools create agones-metrics \
  --cluster=[CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --node-taints agones.dev/agones-metrics=true:NoExecute \
  --node-labels agones.dev/agones-metrics=true \
  --num-nodes=1 \
  --machine-type=e2-standard-4

Replace the following:

[CLUSTER_NAME]: The name of the cluster you created
[COMPUTE_REGION]: The GCP region to create the cluster in, chosen above

Flag explanations:

--cluster: The name of the cluster you created.
--region: The compute region you chose above.
--node-taints: The Kubernetes taints to automatically apply to nodes in this node pool.
--node-labels: The Kubernetes labels to automatically apply to nodes in this node pool.
--num-nodes: The number of nodes per cluster zone. For regional clusters, --num-nodes=1 creates one node in 3 separate zones in the region, giving you faster recovery time in the event of a node failure.
--machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.

(Optional) Creating a node pool for Windows

If you run game servers on Windows, you need to create a dedicated node pool for those servers. Windows Server 2019 (WINDOWS_LTSC_CONTAINERD) is the recommended image for Windows game servers.

Warning

Running GameServers on Windows nodes is currently Alpha. Feel free to file feedback through Github issues.

gcloud container node-pools create windows \
  --cluster=[CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --image-type WINDOWS_LTSC_CONTAINERD \
  --machine-type e2-standard-4 \
  --num-nodes=4

Replace the following:

[CLUSTER_NAME]: The name of the cluster you created
[COMPUTE_REGION]: The GCP region to create the cluster in, chosen above

Flag explanations:

--cluster: The name of the cluster you created.
--region: The compute region you chose above.
--image-type: The image type of the instances in the node pool - WINDOWS_LTSC_CONTAINERD in this case.
--machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.
--num-nodes: The number of nodes per cluster zone. For regional clusters, --num-nodes=1 creates one node in 3 separate zones in the region, giving you faster recovery time in the event of a node failure.

Setting up cluster credentials

gcloud container clusters create configurates credentials for kubectl automatically. If you ever lose those, run:

gcloud container clusters get-credentials [CLUSTER_NAME] --region=[COMPUTE_REGION]

Next Steps

Continue to Install Agones.

Docs: Installing Agones on Google Kubernetes Engine using Terraform

Mon, 01 Jan 0001 00:00:00 +0000

Before you begin

Take the following steps to enable the Kubernetes Engine API:

Visit the Kubernetes Engine page in the Google Cloud Platform Console.
Create or select a project.
Wait for the API and related services to be enabled. This can take several minutes.
Enable billing for your project.

If you are not an existing GCP user, you may be able to enroll for a $300 US Free Trial credit.

Choosing a shell

To complete this quickstart, we can use either Google Cloud Shell or a local shell.

Google Cloud Shell is a shell environment for managing resources hosted on Google Cloud Platform (GCP). Cloud Shell comes preinstalled with the gcloud and kubectl command-line tools. gcloud provides the primary command-line interface for GCP, and kubectl provides the command-line interface for running commands against Kubernetes clusters.

If you prefer using your local shell, you must install the gcloud and kubectl command-line tools in your environment.

Cloud shell

To launch Cloud Shell, perform the following steps:

Go to Google Cloud Platform Console
From the top-right corner of the console, click the Activate Google Cloud Shell button:
A Cloud Shell session opens inside a frame at the bottom of the console. Use this shell to run gcloud and kubectl commands.
Set a compute zone in your geographical region with the following command. The compute zone will be something like us-west1-a. A full list can be found here.
```
gcloud config set compute/zone [COMPUTE_ZONE]
```

Local shell

To install gcloud and kubectl, perform the following steps:

Install the Google Cloud SDK, which includes the gcloud command-line tool.
Initialize some default configuration by running the following command.
- When asked Do you want to configure a default Compute Region and Zone? (Y/n)?, enter Y and choose a zone in your geographical region of choice.
```
gcloud init
```
Install the kubectl command-line tool by running the following command:
```
gcloud components install kubectl
```

Installation

An example configuration can be found here: Terraform configuration with Agones submodule .

Copy this file into a local directory where you will execute the terraform commands.

The GKE cluster created from the example configuration will contain 3 Node Pools:

"default" node pool with "game-server" tag, containing 4 nodes.
"agones-system" node pool for Agones Controller.
"agones-metrics" for monitoring and metrics collecting purpose.

Configurable parameters:

project - your Google Cloud Project ID (required)
name - the name of the GKE cluster (default is “agones-terraform-example”)
agones_version - the version of agones to install (an empty string, which is the default, is the latest version from the Helm repository)
machine_type - machine type for hosting game servers (default is “e2-standard-4”)
node_count - count of game server nodes for the default node pool (default is “4”)
enable_image_streaming - whether or not to enable image streaming for the "default" node pool (default is true)
zone - (Deprecated, use location) the name of the zone you want your cluster to be created in (default is “us-west1-c”)
network - the name of the VPC network you want your cluster and firewall rules to be connected to (default is “default”)
subnetwork - the name of the subnetwork in which the cluster’s instances are launched. (required when using non default network)
log_level - possible values: Fatal, Error, Warn, Info, Debug (default is “info”)
feature_gates - a list of alpha and beta version features to enable. For example, “PlayerTracking=true&ContainerPortAllocation=true”
gameserver_minPort - the lower bound of the port range which gameservers will listen on (default is “7000”)
gameserver_maxPort - the upper bound of the port range which gameservers will listen on (default is “8000”)
gameserver_namespaces - a list of namespaces which will be used to run gameservers (default is ["default"]). For example ["default", "xbox-gameservers", "mobile-gameservers"]
force_update - whether or not to force the replacement/update of resource (default is true, false may be required to prevent immutability errors when updating the configuration)
location - the name of the location you want your cluster to be created in (default is “us-west1-c”)
autoscale - whether you want to enable autoscale for the gameserver nodepool (default is false)
min_node_count - the minimum number of nodes for a nodepool when autoscale is enabled (default is “1”)
max_node_count - the maximum number of nodes for a nodepool when autoscale is enabled (default is “5”)
enable_agones_metrics_nodepool - whether you want to enable "agones-metrics" for the gameserver nodepool (default is false)

Warning

On the lines that read source = "git::https://github.com/googleforgames/agones.git//install/terraform/modules/gke/?ref=main" make sure to change ?ref=main to match your targeted Agones release, as Terraform modules can change between releases.

For example, if you are targeting release-1.40.0, then you will want to have source = "git::https://github.com/googleforgames/agones.git//install/terraform/modules/gke/?ref=release-1.40.0" as your source.

Creating the cluster

In the directory where you created module.tf, run:

terraform init

This will cause terraform to clone the Agones repository and use the ./install/terraform folder as the starting point of the Agones submodule, which contains all necessary Terraform configuration files.

Next, make sure that you can authenticate using gcloud:

gcloud auth application-default login

Option 1: Creating the cluster in the default VPC

To create your GKE cluster in the default VPC just specify the project variable.

terraform apply -var project="<YOUR_GCP_ProjectID>"

Option 2: Creating the cluster in a custom VPC

To create the cluster in a custom VPC you must specify the project, network and subnetwork variables.

terraform apply -var project="<YOUR_GCP_ProjectID>" -var network="<YOUR_NETWORK_NAME>" -var subnetwork="<YOUR_SUBNETWORK_NAME>"

To verify that the cluster was created successfully, set up your kubectl credentials:

gcloud container clusters get-credentials --zone us-west1-c agones-terraform-example

Then check that you have access to the Kubernetes cluster:

kubectl get nodes

You should have 6 nodes in Ready state.

Uninstall the Agones and delete GKE cluster

To delete all resources provisioned by Terraform:

terraform destroy -var project="<YOUR_GCP_ProjectID>"

Next Steps

Confirm Agones is up and running

Docs: Install Agones using YAML

Mon, 01 Jan 0001 00:00:00 +0000

Installing Agones

Warning

Installing Agones with the install.yaml file will use pre-generated, well known TLS certificates stored in this repository for securing Kubernetes webhooks communication.

For production workloads, we strongly recommend using the helm installation which allows you to generate new, unique certificates or provide your own certificates. Alternatively, you can use helm template as described below to generate a custom yaml installation file with unique certificates.

Installing Agones using the pre-generated install.yaml file is the quickest, simplest way to get Agones up and running in your Kubernetes cluster:

kubectl create namespace agones-system
kubectl apply --server-side -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/install/yaml/install.yaml

You can also find the install.yaml in the latest agones-install zip from the releases archive.

Customizing your install

To change the configurable parameters in the install.yaml file, you can use helm template to generate a custom file locally without needing to use helm to install Agones into your cluster.

The following example sets the featureGates and generateTLS helm parameters and creates a customized install-custom.yaml file (note that the pull command was introduced in Helm version 3):

helm pull --untar https://agones.dev/chart/stable/agones-1.40.0.tgz && \
cd agones && \
helm template agones-manual --namespace agones-system  . \
  --set agones.controller.generateTLS=false \
  --set agones.allocator.generateTLS=false \
  --set agones.allocator.generateClientTLS=false \
  --set agones.crds.cleanupOnDelete=false \
  --set agones.featureGates="PlayerTracking=true" \
  > install-custom.yaml

Uninstalling Agones

To uninstall/delete the Agones deployment and delete agones-system namespace:

kubectl delete fleets --all --all-namespaces
kubectl delete gameservers --all --all-namespaces
kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/install/yaml/install.yaml
kubectl delete namespace agones-system

Note: It may take a couple of minutes until all resources described in install.yaml file are deleted.

Next Steps

Confirm Agones is up and running

Docs: Unity Game Server Client SDK

Thu, 13 Jun 2019 10:17:50 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GameServer	✔️
Configuration	Watch	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	❌
Counters	SetCounterCount	❌
Counters	IncrementCounter	❌
Counters	DecrementCounter	❌
Counters	SetCounterCapacity	❌
Counters	GetCounterCapacity	❌
Lists	AppendListValue	❌
Lists	DeleteListValue	❌
Lists	SetListCapacity	❌
Lists	GetListCapacity	❌
Lists	ListContains	❌
Lists	GetListLength	❌
Lists	GetListValues	❌
Player Tracking	GetConnectedPlayers	✔️
Player Tracking	GetPlayerCapacity	✔️
Player Tracking	GetPlayerCount	✔️
Player Tracking	IsPlayerConnected	✔️
Player Tracking	PlayerConnect	✔️
Player Tracking	PlayerDisconnect	✔️
Player Tracking	SetPlayerCapacity	✔️

Additional methods have been added for ease of use:

Connect

Installation

The client SDK code can be manually downloaded and added to your project hierarchy.

It can also be imported into your project via the Unity Package Manager (UPM). To do that, open your project’s manifest.json file, and add the following line to the dependencies section:

{
  "dependencies": {
        "com.googleforgames.agones": "https://github.com/googleforgames/agones.git?path=/sdks/unity",
...

If you want a specific release, the dependency can be pinned to that version. For example:

"com.googleforgames.agones": "https://github.com/googleforgames/agones.git?path=/sdks/unity#v1.40.0",

Download

Download the source directly from GitHub .

Prerequisites

Unity >= 2018.x (.NET 4.x)

Usage

Import this script to your unity project and attach it to GameObject.

To begin working with the SDK, get an instance of it.

var agones = agonesGameObject.GetComponent<Agones.AgonesSdk>();

To connect to the SDK server, either local or when running on Agones, run the async Connect() method. This will wait for up to 30 seconds if the SDK server has not yet started and the connection cannot be made, and will return false if there was an issue connecting.

bool ok = await agones.Connect();

To mark the game server as ready to receive player connections, call the async method Ready().

async void SomeMethod()
{
    bool ok = await agones.Ready();
}

To get the details on the backing GameServer call GameServer().

Will return null if there is an error in retrieving the GameServer record.

var gameserver = await agones.GameServer();

To mark the GameServer as Reserved for a duration call Reserve(TimeSpan duration).

ok = await agones.Reserve(duration);

To mark that the game session is completed and the game server should be shut down call Shutdown().

bool ok = await agones.Shutdown();

Similarly SetAnnotation(string key, string value) and SetLabel(string key, string value) are async methods that perform an action.

And there is no need to call Health(), it is automatically called.

To watch when the backing GameServer configuration changes call WatchGameServer(callback), where the delegate function callback will be executed every time the GameServer configuration changes.

agones.WatchGameServer(gameServer => Debug.Log($"Server - Watch {gameServer}"));

Player Tracking

Warning

The Player Tracking feature is currently Alpha, not enabled by default, and may change in the future.

Use the FeatureGate PlayerTracking to enable and test this feature.

See the Feature Gate documentation for details on how to enable features.

To use alpha features use the AgonesAlphaSDK class.

var agones = agonesGameObject.GetComponent<Agones.AgonesAlphaSdk>();

Alpha: PlayerConnect

This method increases the SDK’s stored player count by one, and appends this playerID to GameServer.Status.Players.IDs. Returns true and adds the playerID to the list of playerIDs if the playerIDs was not already in the list of connected playerIDs.

bool ok = await agones.PlayerConnect(playerId);

Alpha: PlayerDisconnect

This function decreases the SDK’s stored player count by one, and removes the playerID from GameServer.Status.Players.IDs. Will return true and remove the supplied playerID from the list of connected playerIDs if the playerID value exists within the list.

bool ok = await agones.PlayerDisconnect(playerId);

Alpha: SetPlayerCapacity

Update the GameServer.Status.Players.Capacity value with a new capacity.

var capacity = 100;
bool ok = await agones.SetPlayerCapacity(capacity);

Alpha: GetPlayerCapacity

This function retrieves the current player capacity GameServer.Status.Players.Capacity. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

long capacity = await agones.GetPlayerCapacity();

Alpha: GetPlayerCount

Returns the current player count

long count = await agones.GetPlayerCount();

Alpha: IsPlayerConnected

This returns if the playerID is currently connected to the GameServer. This is always accurate, even if the value hasn’t been updated to the GameServer status yet.

bool isConnected = await agones.IsPlayerConnected(playerId);

Alpha: GetConnectedPlayers

This returns a list of the playerIDs that are currently connected to the GameServer.

List<string> players = await agones.GetConnectedPlayers();

Warning

The following code causes deadlock. Do not use a Wait method with the returned Task.

void Deadlock()
{
    Task<bool> t = agones.Shutdown();
    t.Wait(); // deadlock!!!
}

Settings

The properties for the Unity Agones SDK can be found in the Inspector.

Health Interval Second
- Interval of the server sending a health ping to the Agones sidecar. (default: 5)
Health Enabled
- Whether the server sends a health ping to the Agones sidecar. (default: true)
Log Enabled
- Debug Logging Enabled. Debug logging for development of this Plugin. (default: false)

Docs: High Availability Agones

Fri, 10 Feb 2023 00:00:00 +0000

High Availability for Agones Controller

The agones-controller responsibility is split up into agones-controller, which enacts the Agones control loop, and agones-extensions, which acts as a service endpoint for webhooks and the allocation extension API. Splitting these responsibilities allows the agones-extensions pod to be horizontally scaled, making the Agones control plane highly available and more resiliant to disruption.

Multiple agones-controller pods enabled, with a primary controller selected via leader election. Having multiple agones-controller minimizes downtime of the service from pod disruptions such as deployment updates, autoscaler evictions, and crashes.

Extension Pod Configrations

The agones-extensions binary has a similar helm configuration to agones-controller, see here. If you previously overrode agones.controller.* settings, you may need to override the same agones.extensions.* setting.

To change controller.numWorkers to 200 from 100 values and through the use of helm --set, add the follow to the helm command:

Important: This will not have any effect on any extensions values!

 ...
 --set agones.controller.numWorkers=200
 ...

An important configuration to note is the PodDisruptionBudget fields, agones.extensions.pdb.minAvailable and agones.extensions.pdb.maxUnavailable. Currently, the agones.extensions.pdb.minAvailable field is set to 1.

Deployment Considerations

Leader election will automatically be enabled and agones.controller.replicas is > 1. agones.controller.replicas defaults to 2.

The default configuration now deploys 2 agones-controller pods and 2 agones-extensions pods, replacing the previous single agones-controller pod setup. For example:

NAME                                 READY   STATUS    RESTARTS   AGE
agones-allocator-78c6b8c79-h9nqc     1/1     Running   0          23h
agones-allocator-78c6b8c79-l2bzp     1/1     Running   0          23h
agones-allocator-78c6b8c79-rw75j     1/1     Running   0          23h
agones-controller-fbf944f4-vs9xx     1/1     Running   0          23h
agones-controller-fbf944f4-sjk3t     1/1     Running   0          23h
agones-extensions-5648fc7dcf-hm6lk   1/1     Running   0          23h
agones-extensions-5648fc7dcf-qbc6h   1/1     Running   0          23h
agones-ping-5b9647874-2rrl6          1/1     Running   0          27h
agones-ping-5b9647874-rksgg          1/1     Running   0          27h

The number of replicas for agones-extensions can be set using helm variable agones.extensions.replicas, but the default is 2.

We expect the aggregate memory consumption of the pods will be slightly higher than the previous singleton pod, but as the responsibilities are now split across the pods, the aggregate CPU consumption should also be similar.

Feature Design

Please see HA Agones.

Docs: Controlling Disruption

Tue, 24 Jan 2023 20:15:26 +0000

Disruption in Kubernetes

A Pod in Kubernetes may be disrupted for involuntary reasons, e.g. hardware failure, or voluntary reasons, such as when nodes are drained for upgrades.

By default, Agones assumes your game server should never be disrupted voluntarily and configures the Pod appropriately - but this isn’t always the ideal setting. Here we discuss how Agones allows you to control the two most significant sources of voluntary Pod evictions, node upgrades and Cluster Autoscaler, using the eviction API on the GameServer object.

Benefits of Allowing Voluntary Disruption

It’s not always easy to write your game server in a way that allows for disruption, but it can have major benefits:

Compaction of your cluster using Cluster Autoscaler can lead to considerable cost savings for your infrastructure.
Allowing automated node upgrades can save you management toil, and lowers the time it takes to patch security vulnerabilites.

Considerations

When discussing game server pod disruption, it’s important to keep two factors in mind:

TERM signal: Is your game server tolerant of graceful termination? If you wish to support voluntary disruption, your game server must handle the TERM signal (even if it runs to completion after receiving TERM).
Termination Grace Period: After receiving TERM, how long does your game server need to run? If you run to completion after receiving TERM, this is equivalent to the session length - if not, you can think of this as the cleanup time. In general, we bucket the grace period into “less than 10 minutes”, “10 minutes to an hour”, and “greater than an hour”. (See below if you are curious about grace period considerations.)

`eviction` API

The eviction API is specified as part of the GameServerSpec, like:

apiVersion: "agones.dev/v1"
kind: GameServer
metadata:
  name: "simple-game-server"
spec:
  eviction:
    safe: Always
  template:
    [...]

You can set eviction.safe based on your game server’s tolerance for disruption and session length, based on the following diagram:

In words:

Does the game server support TERM and terminate within ten minutes?
- Yes to both: Set safe: Always, and set terminationGracePeriodSeconds to the session length or cleanup time.
- No to either: Does the game server support TERM and terminate within an hour?
  - Yes to both: Set safe: OnUpgrade, and configure terminationGracePeriodSeconds to the session length or cleanup time.
  - No to either: Set safe: Never. If your game server does not terminate within an hour, see below.

Note

To maintain backward compatibility with Agones prior to the introduction of eviction API, if your game server previously configured the cluster-autoscaler.kubernetes.io/safe-to-evict: true annotation, we assume eviction.safe: Always is intended.

Note

GKE Autopilot supports only Never and Always, not OnUpgrade.

What’s special about ten minutes and one hour?

Ten minutes: Cluster Autoscaler respects ten minutes of graceful termination on scale-down. On some cloud products, you can configure --max-graceful-termination-sec to change this, but it is not advised: Cluster Autoscaler is currently only capable of scaling down one node at a time, and larger graceful termination windows slow this down farther (see autoscaler#5079). If the ten minute limit does not apply to you, generally you should choose between safe: Always (for sessions less than an hour), or see below.
One hour: On many cloud products, PodDisruptionBudget can only block node upgrade evictions for a certain period of time - on GKE this is 1h. After that, the PDB is ignored, or the node upgrade fails with an error. Controlling Pod disruption for longer than one hour requires cluster configuration changes outside of Agones - see below.

Considerations for long sessions

Outside of Cluster Autoscaler, the main source of disruption for long sessions is node upgrade. On some cloud products, such as GKE Standard, node upgrades are entirely within your control. On others, such as GKE Autopilot, node upgrade is automatic. Typical node upgrades use an eviction based, rolling recreate strategy, and may not honor PodDisruptionBudget for longer than an hour. See Best Practices for information specific to your cloud product.

Implementation / Under the hood

Each option uses a slightly different permutation of:

the safe-to-evict annotation to block Cluster Autoscaler based eviction, and
the agones.dev/safe-to-evict label selector to select the agones-gameserver-safe-to-evict-false PodDisruptionBudget. This blocks Cluster Autoscaler and (for a limited time) disruption from node upgrades.
- Note that PDBs do influence pod preemption as well, but it’s not guaranteed.

As a quick reference:

evictions.safe setting	`safe-to-evict` pod annotation	`agones.dev/safe-to-evict` label
`Never` (default)	`false`	`false` (matches PDB)
`OnUpgrade`	`false`	`true` (does not match PDB)
`Always`	`true`	`true` (does not match PDB)

Docs: Third Party Examples

Thu, 09 Sep 2021 00:00:00 +0000

Integrations with other projects

Octops/Agones x Open Match - How to implement a matchmaking system using Agones and Open Match

Minetest

Minetest is a free and open-source sandbox video game available for Linux, FreeBSD, Microsoft Windows, MacOS, and Android. Minetest game play is very similar to that of Minecraft. Players explore a blocky 3D world, discover and extract raw materials, craft tools and items, and build structures and landscapes.
Minetest server for Agones is an example of the Minetest server hosted on Kubernetes using Agones. It wraps the Minetest server with a Go binary, and introspects stdout to provide the event hooks for the SDK integration. The wrapper is from Xonotic Example with a few changes to look for the Minetest ready output message.

You will need to download the Minetest client separately to play.

Quilkin

Quilkin is a non-transparent UDP proxy specifically designed for use with large scale multiplayer dedicated game server deployments, to ensure security, access control, telemetry data, metrics and more.
Quilkin with Agones is an example of running Xonotic with Quilkin on an Agones cluster, utilising either the sidecar integration pattern or via the the Quilkin xDS Agones provider with a TokenRouter to provide routing and access control to the allocated GameServer instance.

You will need to download the Xonotic client to interact with the demo.

Shulker

Shulker is a Kubernetes operator for managing complex and dynamic Minecraft infrastructure at scale, including game servers and proxies.
It builds on top Agones GameServer and Fleet primitives to provide simplified abstractions specifically tailored to orchestrating Minecraft workloads.

Shulker requires you to have a genuine Minecraft account. You’ll need to purchase the game to test the “Getting Started” example.

Docs: Matchmaker requires game server process registration

Tue, 27 Jul 2021 00:00:00 +0000

In this scenario, the GameServer process will need to self Allocate when informed by the matchmaker that players are being sent to them.

Warning

This does relinquish control over how GameServers are packed across the cluster to the external matchmaker. It is likely it will not do as good a job at packing and scaling as Agones.

Next Steps:

Read the various references, including the GameServer and Fleet reference materials.
Review the specifics of Health Checking.
See all the commands the Client SDK provides - we only show a few here!
If you aren’t familiar with the term Pod, this should provide a reference.

Docs: Windows Gameservers

Mon, 19 Apr 2021 21:14:19 +0000

Warning

Running GameServers on Windows nodes is currently Alpha, and any feedback would be appreciated.

Prerequisites

The following prerequisites are required to create a GameServer:

A Kubernetes cluster with the UDP port range 7000-8000 open on each node. Creating the firewall.
Agones controller installed in the targeted cluster
kubectl properly configured
Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Agones.

Ensure that you have some nodes to your cluster that are running Windows.

Objectives

Create a GameServer on a Windows node.
Connect to the GameServer.

1. Create a GameServer

Note

Starting with version 0.3, the simple-game-server example is compiled as a multi-arch docker image that will run on both Linux and Windows. To ensure that the game server runs on a Windows node, a nodeSelector of "kubernetes.io/os": windows must be added to the game server specification.

Create a GameServer using the following command:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserver-windows.yaml

You should see a successful output similar to this:

gameserver.agones.dev/simple-game-server-4ss4j created

Verify that the GameServer becomes Ready by running:

kubectl get gameservers

It should look something like this:

NAME                       STATE     ADDRESS       PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   3m

Take a note of the Game Server IP address and ports.

For the full details of the YAML file head to the GameServer Specification Guide

2. Connect to the GameServer

Note

If you have Agones installed on Google Kubernetes Engine, and are using Cloud Shell for your terminal, UDP is blocked. For this step, we recommend SSH’ing into a running VM in your project, such as a Kubernetes node. You can click the ‘SSH’ button on the Google Compute Engine Instances page to do this. Run toolbox on GKE Node to run docker container with tools and then nc command would be available.

You can now communicate with the Game Server:

Note

If you do not have netcat installed (i.e. you get a response of nc: command not found), you can install netcat by running sudo apt install netcat.

If you are on Windows, you can alternatively install netcat on WSL, or download a version of netcat for Windows from nmap.org.

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gameserver again - either the GameServer will be gone completely, or it will be in Shutdown state, on the way to being deleted.

Next Steps

Make a local copy of the simple-game-server fleet configuration , modify it to include a node selector, and use it to go through the Quickstart: Create a Game Server Fleet.
If you want to use your own GameServer container make sure you have properly integrated the Agones SDK.

Docs: Installing Agones on AWS Elastic Kubernetes Service using Terraform

Tue, 21 Jan 2020 00:00:00 +0000

Installation

You can use Terraform to provision your Amazon EKS (Elastic Kubernetes Service) cluster and install Agones on it using the Helm Terraform provider.

An example of the EKS submodule config file can be found here: Terraform configuration with Agones submodule

Copy this file into a separate folder.

Configure your AWS CLI tool CLI configure:

aws configure

Initialise your terraform:

terraform init

Creating Cluster

By editing modules.tf you can change the parameters that you need to. For instance, the - machine_type variable.

Configurable parameters:

cluster_name - the name of the EKS cluster (default is “agones-terraform-example”)
agones_version - the version of agones to install (an empty string, which is the default, is the latest version from the Helm repository)
machine_type - EC2 instance type for hosting game servers (default is “t2.large”)
region - the location of the cluster (default is “us-west-2”)
node_count - count of game server nodes for the default node pool (default is “4”)
log_level - possible values: Fatal, Error, Warn, Info, Debug (default is “info”)
feature_gates - a list of alpha and beta version features to enable. For example, “PlayerTracking=true&ContainerPortAllocation=true”
gameserver_minPort - the lower bound of the port range which gameservers will listen on (default is “7000”)
gameserver_maxPort - the upper bound of the port range which gameservers will listen on (default is “8000”)
gameserver_namespaces - a list of namespaces which will be used to run gameservers (default is ["default"]). For example ["default", "xbox-gameservers", "mobile-gameservers"]
force_update - whether or not to force the replacement/update of resource (default is true, false may be required to prevent immutability errors when updating the configuration)

Now you can create an EKS cluster and deploy Agones on EKS:

terraform apply [-var agones_version="1.40.0"]

After deploying the cluster with Agones, you can get or update your kubeconfig by using:

aws eks --region us-west-2 update-kubeconfig --name agones-cluster

With the following output:

Added new context arn:aws:eks:us-west-2:601646756426:cluster/agones-cluster to /Users/user/.kube/config

Switch kubectl context to the recently created one:

kubectl config use-context arn:aws:eks:us-west-2:601646756426:cluster/agones-cluster

Check that you are authenticated against the recently created Kubernetes cluster:

kubectl get nodes

Uninstall the Agones and delete EKS cluster

Run the following commands to delete all Terraform provisioned resources:

terraform destroy -target module.helm_agones.helm_release.agones -auto-approve && sleep 60
terraform destroy

Note

There is an issue with the AWS Terraform provider: https://github.com/terraform-providers/terraform-provider-aws/issues/9101 Due to this issue you should remove helm release first (as stated above), otherwise terraform destroy will timeout and never succeed. Remove all created resources manually in that case, namely: 3 Auto Scaling groups, EKS cluster, and a VPC with all dependent resources.

Docs: Third Party Podcasts

Wed, 15 Jan 2020 00:00:00 +0000

2019

Nov 6 - [Google Cloud Platform Podcast: Supersolid with Kami May] (https://gcppodcast.com/post/episode-202-supersolid-with-kami-may/)

2018

Oct 23 - [Kubernetes Podcast: Agones, with Cyril Tovena and Mark Mandel] (https://kubernetespodcast.com/episode/026-agones/)
Aug 15 - [Google Cloud Platform Podcast: Agones with Mark Mandel and Cyril Tovena] (https://gcppodcast.com/post/episode-142-agones-with-mark-mandel-and-cyril-tovena/)

Docs: Fleet Updates

Tue, 27 Aug 2019 03:58:19 +0000

Rolling Update Strategy

When Fleets are edited and updated, the default strategy of Agones is to roll the new version of the GameServer out to the entire Fleet, in a step by step increment and decrement by adding a chunk of the new version and removing a chunk of the current set of GameServers.

This is done while ensuring that Allocated GameServers are not deleted until they are specifically shutdown through the game servers SDK, as they are expected to have players on them.

You can see this in the Fleet.Spec.Strategy reference, with controls for how much of the Fleet is incremented and decremented at one time:

  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%

So when a Fleet is edited (any field other than replicas, see note below), either through kubectl edit/apply or via the Kubernetes API, this performs the following operations:

Adds the maxSurge number of GameServers to the Fleet.
Shutdown the maxUnavailable number of GameServers in the Fleet, skipping Allocated GameServers.
Repeat above steps until all the previous GameServer configurations have been Shutdown and deleted.

By default, a Fleet will wait for new GameServers to become Ready during a Rolling Update before continuing to shutdown additional GameServers, only counting GameServers that are Ready as being available when calculating the current maxUnavailable value which controls the rate at which GameServers are updated. This ensures that a Fleet cannot accidentally have zero GameServers in the Ready state if something goes wrong during a Rolling Update or if GameServers have a long delay before moving to the Ready state.

Note

When Fleet update contains only changes to the replicas parameter, then new GameServers will be created/deleted straight away, which means in that case maxSurge and maxUnavailable parameters for a RollingUpdate will not be used. The RollingUpdate strategy takes place when you update spec parameters other than replicas.

If you are using a Fleet which is scaled by a FleetAutoscaler, read the Fleetautoscaler guide for more details on how RollingUpdates with FleetAutoscalers need to be implemented.

You could also check the behaviour of the Fleet with a RollingUpdate strategy on a test Fleet to preview your upcoming updates. Use kubectl describe fleet to track scaling events in a Fleet.

Recreate Strategy

This is an optimal Fleet update strategy if you want to replace all GameServers that are not Allocated with a new version as quickly as possible.

You can see this in the Fleet.Spec.Strategy reference:

  strategy:
    type: Recreate

So when a Fleet is edited, either through kubectl edit/apply or via the Kubernetes API, this performs the following operations:

Shutdown all GameServers in the Fleet that are not currently Allocated.
Create the same number of the new version of the GameServers that were previously deleted.
Repeat above steps until all the previous GameServer configurations have been Shutdown and deleted.

Two (or more) Fleets Strategy

If you want very fine-grained control over the rate that new versions of a GameServer configuration is rolled out, or if you want to do some version of A/B testing or smoke test between different versions, running two (or more) Fleets at the same time is a good solution for this.

To do this, create a second Fleet inside your cluster, starting with zero replicas. From there you can scale this newer Fleet up and the older Fleet down as required by your specific rollout strategy.

This also allows you to rollback if issues arise with the newer version, as you can delete the newer Fleet and scale up the old Fleet to its previous levels, resulting in minimal impact to the players.

Note

For GameServerAllocation, you will need to have at least a single shared label between the GameServers in each Fleet.

GameServerAllocation Across Fleets

Since GameServerAllocation is powered by label selectors, it is possible to allocate across multiple fleets, and/or give preference to particular sets of GameServers over others. You can see details of this in the GameServerAllocation reference.

In a scenario where a new v2 version of a Fleet is being slowly scaled up in a separate Fleet from the previous v1 Fleet, we can specify that we prefer allocation to occur from the v2 Fleet, and if none are available, fallback to the v1 Fleet, like so:

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        agones.dev/fleet: v2
    - matchLabels:
        game: my-awesome-game

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  # Deprecated, use field selectors instead.
  required:
    matchLabels:
      game: my-awesome-game
  # Deprecated, use field selectors instead.
  preferred:
    - matchLabels:
        agones.dev/fleet: v2

In this example, all GameServers have the label game: my-awesome-game, so the Allocation will search across both Fleets through that mechanism. The selectors label matching selector tells the allocation system to first search all GameServers with the v2 Fleet label, and if not found, search through the rest of the set.

The above GameServerAllocation can then be used while you scale up the v2 Fleet and scale down the original Fleet at the rate that you deem fit for your specific rollout.

Notifying GameServers on Fleet Update/Downscale

When Allocated GameServers are utilised for a long time, such as a Lobby GameServer, or a GameServer that is being reused multiple times in a row, it can be useful to notify an Allocated GameServer process when its backing Fleet has been updated. When an update occurs, the Allocated GameServer, may want to actively perform a graceful shutdown and release its resources such that it can be replaced by a new version, or similar actions.

To do this, we provide the ability to apply a user-provided set of labels and/or annotations to the Allocated GameServers when a Fleet update occurs that updates its GameServer template, or generally causes the Fleet replica count to drop below the number of currently Allocated GameServers.

This provides two useful capabilities:

The GameServer SDK.WatchGameServer() command can be utilised to react to this annotation and/or label change to indicate the Fleet system change, and the game server binary could execute code accordingly.
This can also be used to proactively update GameServer labels, to effect change in allocation strategy - such as preferring the newer GameServers when allocating, but falling back to the older version if there aren’t enough of the new ones yet spun up.

The labels and/or annotations are applied to GameServers in a Fleet in the order designated by their configured Fleet scheduling.

Example yaml configuration:

apiVersion: "agones.dev/v1"
kind: Fleet
metadata:
  name: simple-game-server
spec:
  replicas: 2
  allocationOverflow: # This specifies which annotations and/or labels are applied
    labels:
      mykey: myvalue
      version: "" # empty an existing label value, so it's no longer in the allocation selection
    annotations:
      event: overflow
  template:
    spec:
      ports:
        - name: default
          containerPort: 7654
      template:
        spec:
          containers:
            - name: simple-game-server
              image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.27

See the Fleet reference for more details.

Note

This works the same across Fleet resizing and Rolling/Recreate Updates, in that the implementation responds to the underlying GameServerSet’s replicas being shrunk to a value smaller than the number of Allocated GameServers it controls. Therefore, this functionality works equally well with a rolling update as it does with an update strategy that requires at least two Fleets.

Docs: Fleet Specification

Thu, 03 Jan 2019 03:58:52 +0000

To allocate a GameServer from a Fleet, use a GameServerAllocation.

Like any other Kubernetes resource you describe a Fleet’s desired state via a specification written in YAML or JSON to the Kubernetes API. The Agones controller will then change the actual state to the desired state.

A full Fleet specification is available below and in the example folder for reference :

apiVersion: "agones.dev/v1"
kind: Fleet
# Fleet Metadata
# https://v1-28.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#objectmeta-v1-meta
metadata:
  name: fleet-example
spec:
  # the number of GameServers to keep Ready or Allocated in this Fleet
  replicas: 2
  # defines how GameServers are organised across the cluster.
  # Options include:
  # "Packed" (default) is aimed at dynamic Kubernetes clusters, such as cloud providers, wherein we want to bin pack
  # resources
  # "Distributed" is aimed at static Kubernetes clusters, wherein we want to distribute resources across the entire
  # cluster
  scheduling: Packed
  # a GameServer template - see:
  # https://agones.dev/site/docs/reference/gameserver/ for all the options
  strategy:
    # The replacement strategy for when the GameServer template is changed. Default option is "RollingUpdate",
    # "RollingUpdate" will increment by maxSurge value on each iteration, while decrementing by maxUnavailable on each
    # iteration, until all GameServers have been switched from one version to another.
    # "Recreate" terminates all non-allocated GameServers, and starts up a new set with the new details to replace them.
    type: RollingUpdate
    # Only relevant when `type: RollingUpdate`
    rollingUpdate:
      # the amount to increment the new GameServers by. Defaults to 25%
      maxSurge: 25%
      # the amount to decrements GameServers by. Defaults to 25%
      maxUnavailable: 25%
  # Labels and/or Annotations to apply to overflowing GameServers when the number of Allocated GameServers is more
  # than the desired replicas on the underlying `GameServerSet`
  allocationOverflow:
    labels:
      mykey: myvalue
      version: "" # empty an existing label value
    annotations:
      otherkey: setthisvalue
  #
  # [Stage:Alpha]
  # [FeatureFlag:CountsAndLists]
  # Which gameservers in the Fleet are most important to keep around - impacts scale down logic.
  # priorities:
  # - type: Counter # Sort by a “Counter”
  #   key: rooms # The name of the Counter. No impact if no GameServer found.
  #   order: Descending # Default is "Ascending" so smaller available capacity will be removed first on down scaling.
  # - type: List # Sort by a “List”
  #   key: players # The name of the List. No impact if no GameServer found.
  #   order: Ascending # Default is "Ascending" so smaller available capacity will be removed first on down scaling.
  #  
  template:
    # GameServer metadata
    metadata:
      labels:
        foo: bar
    # GameServer specification
    spec:
      ports:
      - name: default
        portPolicy: Dynamic
        containerPort: 26000
      health:
        initialDelaySeconds: 30
        periodSeconds: 60
      # Parameters for game server sidecar
      sdkServer:
        logLevel: Info
        grpcPort: 9357
        httpPort: 9358
      #
      # [Stage:Alpha]
      # [FeatureFlag:CountsAndLists]
      # Counts and Lists provides the configuration for generic (player, room, etc.) tracking features.
      # counters:
      #   rooms:
      #     count: 0 # Initial Value
      #     capacity: 10
      # lists:
      #   players:
      #     values: []
      #
      # The GameServer's Pod template
      template:
        spec:
          containers:
          - name: simple-game-server
            image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32

Since Agones defines a new Custom Resources Definition (CRD) we can define a new resource using the kind Fleet with the custom group agones.dev and API version v1.

You can use the metadata field to target a specific namespaces but also attach specific annotations and labels to your resource. This is a very common pattern in the Kubernetes ecosystem.

The length of the name field of the fleet should be at most 63 characters.

The spec field is the actual Fleet specification and it is composed as follow:

replicas is the number of GameServers to keep Ready or Allocated in this Fleet
scheduling defines how GameServers are organised across the cluster. Affects backing Pod scheduling, as well as scale down mechanics. “Packed” (default) is aimed at dynamic Kubernetes clusters, such as cloud providers, wherein we want to bin pack resources. “Distributed” is aimed at static Kubernetes clusters, wherein we want to distribute resources across the entire cluster. See Scheduling and Autoscaling for more details.
strategy is the GameServer replacement strategy for when the GameServer template is edited.
- type is replacement strategy for when the GameServer template is changed. Default option is “RollingUpdate”, but “Recreate” is also available.
  - RollingUpdate will increment by maxSurge value on each iteration, while decrementing by maxUnavailable on each iteration, until all GameServers have been switched from one version to another.
  - Recreate terminates all non-allocated GameServers, and starts up a new set with the new GameServer configuration to replace them.
- rollingUpdate is only relevant when type: RollingUpdate
  - maxSurge is the amount to increment the new GameServers by. Defaults to 25%
  - maxUnavailable is the amount to decrements GameServers by. Defaults to 25%
allocationOverflow (Beta, requires FleetAllocationOverflow flag) The labels and/or Annotations to apply to GameServers when the number of Allocated GameServers exceeds the desired replicas in the underlying GameServerSet.
- labels the map of labels to be applied
- annotations the map of annotations to be applied
- Fleet's Scheduling Strategy: The GameServers associated with the GameServerSet are sorted based on either Packed or Distributed strategy.
  - Packed: Agones maximizes resource utilization by trying to populate nodes that are already in use before allocating GameServers to other nodes.
  - Distributed: Agones employs this strategy to spread out GameServer allocations, ensuring an even distribution of GameServers across the available nodes.
priorities: (Alpha, requires CountsAndLists feature flag): Defines which GameServers in the Fleet are most important to keep around - impacts scale down logic.
- type: Sort by a “Counter” or a “List”.
- key: The name of the Counter or List. If not found on the GameServer, has no impact.
- order: Order: Sort by “Ascending” or “Descending”. “Descending” a bigger available capacity is preferred. “Ascending” would be smaller available capacity is preferred.
template a full GameServer configuration template. See the GameServer reference for all available fields.

Fleet Scale Subresource Specification

Scale subresource is defined for a Fleet. Please refer to Kubernetes docs.

You can use the following command to scale the fleet with name simple-game-server:

kubectl scale fleet simple-game-server --replicas=10

fleet.agones.dev/simple-game-server scaled

You can also use Kubernetes API to get or update the Replicas count:

curl http://localhost:8001/apis/agones.dev/v1/namespaces/default/fleets/simple-game-server/scale

{
  "kind": "Scale",
  "apiVersion": "autoscaling/v1",
  "metadata": {
    "name": "simple-game-server",
    "namespace": "default",
    "selfLink": "/apis/agones.dev/v1/namespaces/default/fleets/simple-game-server/scale",
    "uid": "4dfaa310-2566-11e9-afd1-42010a8a0058",
    "resourceVersion": "292652",
    "creationTimestamp": "2019-01-31T14:41:33Z"
  },
  "spec": {
    "replicas": 10
  },
  "status": {
    "replicas": 10
  }

Also exposing a Scale subresource would allow you to configure HorizontalPodAutoscaler and PodDisruptionBudget for a fleet in the future. However these features have not been tested, and are not currently supported - but if you are looking for these features, please be sure to let us know in the ticket.

Docs: GameServer Health Checking

Thu, 03 Jan 2019 01:20:19 +0000

Disabling Health Checking

By default, health checking is enabled, but it can be turned off by setting the spec.health.disabled property to true.

SDK API

The Health() function on the SDK object needs to be called at an interval less than the spec.health.periodSeconds threshold time to be considered before it will be considered a failure.

The health check will also need to have not been called a consecutive number of times (spec.health.failureThreshold), giving it a chance to heal if it there is an issue.

Health Failure Strategy

The following is the process for what happens to a GameServer when it is unhealthy.

If the GameServer container exits with an error before the GameServer moves to Ready then, it is restarted as per the restartPolicy (which defaults to “Always”).
If the GameServer fails health checking at any point, then it doesn’t restart, but moves to an Unhealthy state.
If the GameServer container exits while in Ready, Allocated or Reserved state, it will be restarted as per the restartPolicy (which defaults to “Always”, since RestartPolicy is a Pod wide setting), but will immediately move to an Unhealthy state.
If the SDK sidecar fails, then it will be restarted, assuming the RestartPolicy is Always/OnFailure.

Fleet Management of Unhealthy GameServers

If a GameServer moves into an Unhealthy state when it is not part of a Fleet, the GameServer will remain in the Unhealthy state until explicitly deleted. This is useful for debugging Unhealthy GameServers, or if you are creating your own GameServer management layer, you can explicitly choose what to do if a GameServer becomes Unhealthy.

If a GameServer is part of a Fleet, the Fleet management system will delete any Unhealthy GameServers and immediately replace them with a brand new GameServer to ensure it has the configured number of Replicas.

Configuration Reference

  # Health checking for the running game server
  health:
    # Disable health checking. defaults to false, but can be set to true
    disabled: false
    # Number of seconds after the container has started before health check is initiated. Defaults to 5 seconds
    initialDelaySeconds: 5
    # If the `Health()` function doesn't get called at least once every period (seconds), then
    # the game server is not healthy. Defaults to 5
    periodSeconds: 5
    # Minimum consecutive failures for the health probe to be considered failed after having succeeded.
    # Defaults to 3. Minimum value is 1
    failureThreshold: 3

See the full GameServer example for more details

Example

C++

For a configuration that requires a health ping every 5 seconds, the example below sends a request every 2 seconds to be sure that the GameServer is under the threshold.

void doHealth(agones::SDK *sdk) {
    while (true) {
        if (!sdk->Health()) {
            std::cout << "Health ping failed" << std::endl;
        } else {
            std::cout << "Health ping sent" << std::endl;
        }
        std::this_thread::sleep_for(std::chrono::seconds(2));
    }
}

int main() {
    agones::SDK *sdk = new agones::SDK();
    bool connected = sdk->Connect();
    if (!connected) {
        return -1;
    }
    std::thread health (doHealth, sdk);

    // ...  run the game server code

}

Full Game Server

Also look in the examples directory.

Docs: C++ Game Server Client SDK

Wed, 02 Jan 2019 10:17:50 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GameServer	✔️
Configuration	WatchGameServer	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	❌
Counters	SetCounterCount	❌
Counters	IncrementCounter	❌
Counters	DecrementCounter	❌
Counters	SetCounterCapacity	❌
Counters	GetCounterCapacity	❌
Lists	AppendListValue	❌
Lists	DeleteListValue	❌
Lists	SetListCapacity	❌
Lists	GetListCapacity	❌
Lists	ListContains	❌
Lists	GetListLength	❌
Lists	GetListValues	❌

Installation

Download

Download the source from the Releases Page or directly from GitHub .

Building the Libraries from source

CMake is used to build SDK for all supported platforms (Linux/Window/macOS).

Prerequisites

CMake >= 3.15.0
Git
C++17 compiler

Dependencies

Agones SDK only depends on the gRPC library.

Warning

Prior to Agones release 1.39.0 if the gRPC dependency was not found locally installed, the CMake system would install the supported gRPC version for you. Unfortunately this process was very brittle and often broke with gRPC updates, therefore this functionality has been removed, and a manual installation of gRPC is now required.

This version of the Agones C++ SDK has been tested with gRPC 1.57.1. To install it from source follow the instructions.

It may also be available from your system’s package manager, but that may not align with the supported gRPC version, so use at your own risk.

Linux / MacOS

mkdir -p .build
cd .build
cmake .. -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=./install
cmake --build . --target install

Windows

Building with Visual Studio:

md .build
cd .build
cmake .. -G "Visual Studio 15 2017 Win64" -DCMAKE_INSTALL_PREFIX=./install
cmake --build . --config Release --target install

Building with NMake

md .build
cd .build
cmake .. -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install
cmake --build . --target install

CMAKE_INSTALL_PREFIX may be skipped if it is OK to install Agones SDK to a default location (usually /usr/local or c:/Program Files/Agones).

CMake option -Wno-dev is specified to suppress CMP0048 deprecation warning for gRPC build.

Usage

Using SDK

In CMake-based projects it’s enough to specify a folder where SDK is installed with CMAKE_PREFIX_PATH and use find_package(agones CONFIG REQUIRED) command. For example: cpp-simple . It may be useful to disable some protobuf warnings in your project.

Usage

The C++ SDK is specifically designed to be as simple as possible, and deliberately doesn’t include any kind of singleton management, or threading/asynchronous processing to allow developers to manage these aspects as they deem appropriate for their system.

We may consider these types of features in the future, depending on demand.

To begin working with the SDK, create an instance of it:

agones::SDK *sdk = new agones::SDK();

To connect to the SDK server, either local or when running on Agones, run the sdk->Connect() method. This will block for up to 30 seconds if the SDK server has not yet started and the connection cannot be made, and will return false if there was an issue connecting.

bool ok = sdk->Connect();

To send a health check call sdk->Health(). This is a synchronous request that will return false if it has failed in any way. Read GameServer Health Checking for more details on the game server health checking strategy.

bool ok = sdk->Health();

To mark the game server as ready to receive player connections, call sdk->Ready(). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Ready();
if (!status.ok()) { ... }

To mark the game server as allocated, call sdk->Allocate(). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Allocate();
if (!status.ok()) { ... }

To mark the game server as reserved, call sdk->Reserve(seconds). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Reserve(std::chrono::seconds(N));
if (!status.ok()) { ... }

To mark that the game session is completed and the game server should be shut down call sdk->Shutdown(). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Shutdown();
if (!status.ok()) { ... }

To set a Label on the backing GameServer call sdk->SetLabel(key, value). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->SetLabel("test-label", "test-value");
if (!status.ok()) { ... }

To set an Annotation on the backing GameServer call sdk->SetAnnotation(key, value). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

status = sdk->SetAnnotation("test-annotation", "test value");
if (!status.ok()) { ... }

To get the details on the backing GameServer call sdk->GameServer(&gameserver), passing in a agones::dev::sdk::GameServer* to push the results of the GameServer configuration into.

This function will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

agones::dev::sdk::GameServer gameserver;
grpc::Status status = sdk->GameServer(&gameserver);
if (!status.ok()) {...}

To get updates on the backing GameServer as they happen, call sdk->WatchGameServer([](const agones::dev::sdk::GameServer& gameserver){...}).

This will call the passed in std::function synchronously (this is a blocking function, so you may want to run it in its own thread) whenever the backing GameServer is updated.

sdk->WatchGameServer([](const agones::dev::sdk::GameServer& gameserver){
  std::cout << "GameServer Update:\n"                                 //
            << "\tname: " << gameserver.object_meta().name() << "\n"  //
            << "\tstate: " << gameserver.status().state() << "\n"
            << std::flush;
});

Next Steps

Read the SDK Overview to review all SDK functionality
Look at the C++ example for a full build template .
Check out sdk.h .

Docs: Quickstart: Create a Game Server Fleet

Wed, 02 Jan 2019 06:42:20 +0000

Prerequisites

The following prerequisites are required to create a GameServer:

A Kubernetes cluster with the UDP port range 7000-8000 open on each node. Creating the firewall.
Agones controller installed in the targeted cluster
kubectl properly configured
Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Agones.

While not required, you may wish to go through the Create a Game Server quickstart before this one.

Objectives

Create a Fleet in Kubernetes using an Agones custom resource.
Scale the Fleet up from its initial configuration.
Request a GameServer allocation from the Fleet to play on.
Connect to the allocated GameServer.
Deploy a new GameServer configuration to the Fleet.

1. Create a Fleet

Let’s create a Fleet using the following command:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/fleet.yaml

You should see a successful output similar to this :

fleet.agones.dev/simple-game-server created

This has created a Fleet record inside Kubernetes, which in turn creates two warm GameServers that are available to be allocated for a game session.

kubectl get fleet

It should look something like this:

NAME                 SCHEDULING   DESIRED   CURRENT   ALLOCATED   READY     AGE
simple-game-server   Packed       2         3         0           2         9m

You can also see the GameServers that have been created by the Fleet by running kubectl get gameservers, the GameServer will be prefixed by simple-game-server.

NAME                             STATE     ADDRESS            PORT   NODE      AGE
simple-game-server-llg4x-rx6rc   Ready     192.168.122.205    7752   minikube   9m
simple-game-server-llg4x-v6g2r   Ready     192.168.122.205    7623   minikube   9m

For the full details of the YAML file head to the Fleet Specification Guide

Note

The game servers deployed from a Fleet resource will be deployed in the same namespace. The above example omits specifying a namespace, which implies both the Fleet and the associated GameServer resources will be deployed to the default namespace.

2. Fetch the Fleet status

Let’s wait for the two GameServers to become ready.

watch kubectl describe fleet simple-game-server

Name:         simple-game-server
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"agones.dev/v1","kind":"Fleet","metadata":{"annotations":{},"name":"simple-game-server","namespace":"default"},"spec":{"replicas":2,...
API Version:  agones.dev/v1
Kind:         Fleet
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-07-01T18:55:35Z
  Generation:          1
  Resource Version:    24685
  Self Link:           /apis/agones.dev/v1/namespaces/default/fleets/simple-game-server
  UID:                 56710a91-7d60-11e8-b2dd-08002703ef08
Spec:
  Replicas:  2
  Strategy:
    Rolling Update:
      Max Surge:        25%
      Max Unavailable:  25%
    Type:               RollingUpdate
  Template:
    Metadata:
      Creation Timestamp:  <nil>
    Spec:
      Health:
      Ports:
        Container Port:  7654
        Name:            default
        Port Policy:     Dynamic
      Template:
        Metadata:
          Creation Timestamp:  <nil>
        Spec:
          Containers:
            Image:  us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32
            Name:   simple-game-server
            Resources:
Status:
  Allocated Replicas:  0
  Ready Replicas:      2
  Replicas:            2
Events:
  Type    Reason                 Age   From              Message
  ----    ------                 ----  ----              -------
  Normal  CreatingGameServerSet  13s   fleet-controller  Created GameServerSet simple-game-server-wlqnd

If you look towards the bottom, you can see there is a section of Status > Ready Replicas which will tell you how many GameServers are currently in a Ready state. After a short period, there should be 2 Ready Replicas.

3. Scale up the Fleet

Let’s scale up the Fleet from 2 replicates to 5.

Run kubectl scale fleet simple-game-server --replicas=5 to change Replicas count from 2 to 5.

If we now run kubectl get gameservers we should see 5 GameServers prefixed by simple-game-server.

NAME                             STATE    ADDRESS           PORT    NODE       AGE
simple-game-server-sdhzn-kcmh6   Ready    192.168.122.205   7191    minikube   52m
simple-game-server-sdhzn-pdpk5   Ready    192.168.122.205   7752    minikube   53m
simple-game-server-sdhzn-r4d6x   Ready    192.168.122.205   7623    minikube   52m
simple-game-server-sdhzn-wng5k   Ready    192.168.122.205   7709    minikube   53m
simple-game-server-sdhzn-wnhsw   Ready    192.168.122.205   7478    minikube   52m

4. Allocate a Game Server from the Fleet

Since we have a fleet of warm gameservers, we need a way to request one of them for usage, and mark that it has players accessing it (and therefore, it should not be deleted until they are finished with it).

Note

In production, you would likely do the following through a Kubernetes API call, but we can also do this through kubectl as well, and ask it to return the response in yaml so that we can see what has happened.

We can do the allocation of a GameServer for usage through a GameServerAllocation, which will both return to us the details of a GameServer (assuming one is available), and also move it to the Allocated state, which demarcates that it has players on it, and should not be removed until SDK.Shutdown() is called, or it is manually deleted.

It is worth noting that there is nothing specific that ties a GameServerAllocation to a fleet. A GameServerAllocation uses a label selector to determine what group of GameServers it will attempt to allocate out of. That being said, a Fleet and GameServerAllocation are often used in conjunction.

This example uses the label selector to specifically target the simple-game-server fleet that we just created.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

For the full details of the YAML file head to the GameServerAllocation Specification Guide

You should get back a response that looks like the following:

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
metadata:
  creationTimestamp: 2019-02-19T02:13:12Z
  name: simple-game-server-dph9b-hfk24
  namespace: default
spec:
  metadata: {}
  required:
    matchLabels:
      agones.dev/fleet: simple-game-server
  scheduling: Packed
status:
  address: 192.168.122.152
  gameServerName: simple-game-server-dph9b-hfk24
  nodeName: minikube
  ports:
  - name: default
    port: 7714
  state: Allocated

If you look at the status section, there are several things to take note of. The state value will tell if a GameServer was allocated or not. If a GameServer could not be found, this will be set to UnAllocated. If there are too many concurrent requests overwhelmed the system, state will be set to Contention even though there are available GameServers.

However, we see that the status.state value was set to Allocated. This means you have been successfully allocated a GameServer out of the fleet, and you can now connect your players to it!

You can see various immutable details of the GameServer in the status - the address, ports and the name of the GameServer, in case you want to use it to retrieve more details.

We can also check to see how many GameServers you have Allocated vs Ready with the following command (“gs” is shorthand for “gameserver”).

kubectl get gs

This will get you a list of all the current GameServers and their Status.State.

NAME                             STATE       ADDRESS           PORT   NODE      AGE
simple-game-server-sdhzn-kcmh6   Ready       192.168.122.205   7191   minikube  52m
simple-game-server-sdhzn-pdpk5   Ready       192.168.122.205   7752   minikube  53m
simple-game-server-sdhzn-r4d6x   Allocated   192.168.122.205   7623   minikube  52m
simple-game-server-sdhzn-wng5k   Ready       192.168.122.205   7709   minikube  53m
simple-game-server-sdhzn-wnhsw   Ready       192.168.122.205   7478   minikube  52m

Note

GameServerAllocations are create only and not stored for performance reasons, so you won’t be able to list them after they have been created - but you can see their effects on GameServers

A handy trick for checking to see how many GameServers you have Allocated vs Ready, run the following:

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE       ADDRESS          PORT   NODE        AGE
simple-game-server-tfqn7-c9tqz   Ready       192.168.39.150   7136   minikube    52m
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148   minikube    53m
simple-game-server-tfqn7-p8wnl   Ready       192.168.39.150   7453   minikube    52m
simple-game-server-tfqn7-t6bwp   Ready       192.168.39.150   7228   minikube    53m
simple-game-server-tfqn7-wkb7b   Ready       192.168.39.150   7226   minikube    52m

5. Scale down the Fleet

Not only can we scale our fleet up, but we can scale it down as well.

The nice thing about Agones is that it is smart enough to know when GameServers have been moved to Allocated and will automatically leave them running on scale down – as we assume that players are playing on this game server, and we shouldn’t disconnect them!

Let’s scale down our Fleet to 0 (yep! you can do that!), and watch what happens.

Run kubectl scale fleet simple-game-server --replicas=0 to change Replicas count from 5 to 0.

It may take a moment for all the GameServers to shut down, so let’s watch them all and see what happens:

watch kubectl get gs

Eventually, one by one they will be removed from the list, and you should simply see:

NAME                             STATUS      ADDRESS          PORT    NODE       AGE
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148    minikube   55m

That lone Allocated GameServer is left all alone, but still running!

If you would like, try editing the Fleet configuration replicas field and watch the list of GameServers grow and shrink.

6. Connect to the GameServer

Since we’ve only got one allocation, we’ll just grab the details of the IP and port of the only allocated GameServer:

kubectl get gameservers | grep Allocated | awk '{print $3":"$4 }'

This should output your Game Server IP address and port. (eg 10.130.65.208:7936)

You can now communicate with the GameServer:

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gs | grep State again - either the GameServer will be replaced with a new, Ready GameServer , or it will be in Shutdown state, on the way to being deleted.

Since we are running a Fleet, Agones will always do it’s best to ensure there are always the configured number of GameServers in the pool in either a Ready or Allocated state.

7. Deploy a new version of the GameServer on the Fleet

We can also change the configuration of the GameServer of the running Fleet, and have the changes roll out, without interrupting the currently Allocated GameServers.

Let’s take this for a spin! Run kubectl scale fleet simple-game-server --replicas=5 to return Replicas count back to 5.

Let’s also allocate ourselves a GameServer:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

We should now have four Ready GameServers and one Allocated.

We can check this by running kubectl get gs.

NAME                             STATE       ADDRESS          PORT   NODE       AGE
simple-game-server-tfqn7-c9tz7   Ready       192.168.39.150   7136   minikube   5m
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148   minikube   5m
simple-game-server-tfqn7-n0wnl   Ready       192.168.39.150   7453   minikube   5m
simple-game-server-tfqn7-hiiwp   Ready       192.168.39.150   7228   minikube   5m
simple-game-server-tfqn7-w8z7b   Ready       192.168.39.150   7226   minikube   5m

In production, we’d likely be changing a containers > image configuration to update our Fleet to run a new game server process, but to make this example simple, change containerPort from 7654 to 6000.

Run kubectl edit fleet simple-game-server, and make the necessary changes, and then save and exit your editor.

This will start the deployment of a new set of GameServers running with a Container Port of 6000.

Warning

This will make it such that you can no longer connect to the simple-game-server game server.

Run kubectl describe gs | grep "Container Port" until you can see that there is one with a containerPort of 7654, which is the Allocated GameServer, and four instances with a containerPort of 6000 which is the new configuration. You can also run kubectl get gs and look at the Age column to see that one GameServer is much older than the other four.

You have now deployed a new version of your game!

Next Steps

Have a look at the GameServerAllocation specification, and see how the extra functionality can enable smoke testing, server information communication, and more.
You can now create a fleet autoscaler to automatically resize your fleet based on the actual usage. See Create a Fleet Autoscaler.
Have a look at the GameServer Integration Patterns, to give you a set of examples on how all the pieces fit together with your matchmaker and other systems.
Or if you want to try to use your own GameServer container make sure you have properly integrated the Agones SDK.
If you would like to learn how to programmatically allocate a Game Server from the fleet, see how to Access Agones via the Kubernetes API or alternatively use the Allocator Service, depending on your needs.

Docs: Amazon Elastic Kubernetes Service

Mon, 01 Jan 0001 00:00:00 +0000

Create your EKS Cluster using the Getting Started Guide.

Possible steps are the following:

Create new IAM role for cluster management.
Run aws configure to authorize your awscli with proper AWS Access Key ID and AWS Secret Access Key.
Create an example cluster:

eksctl create cluster \
--name prod \
--version 1.29 \
--nodegroup-name standard-workers \
--node-type t3.medium \
--nodes 3 \
--nodes-min 3 \
--nodes-max 4

Note

EKS does not use the normal Kubernetes networking since it is incompatible with Amazon VPC networking.

Allowing UDP Traffic

For Agones to work correctly, we need to allow UDP traffic to pass through to our EKS cluster worker nodes. To achieve this, we must update the workers’ nodepool SG (Security Group) with the proper rule. A simple way to do that is:

Log in to the AWS Management Console
Go to the VPC Dashboard and select Security Groups
Find the Security Group for the workers nodepool, which will be named something like eksctl-[cluster-name]-nodegroup-[cluster-name]-workers/SG
Select Inbound Rules
Edit Rules to add a new Custom UDP Rule with a 7000-8000 port range and an appropriate Source CIDR range (0.0.0.0/0 allows all traffic)

Next Steps

Continue to Install Agones.

Docs: Installing Agones on Azure Kubernetes Service using Terraform

Mon, 01 Jan 0001 00:00:00 +0000

Installation

Install az utility by following these instructions.

The example of AKS submodule configuration could be found here: Terraform configuration with Agones submodule

Copy module.tf file into a separate folder.

az login

Configure your terraform:

terraform init

Create a service principal and configure its access to Azure resources:

az ad sp create-for-rbac

Now you can deploy your cluster (use values from the above command output):

terraform apply -var client_id="<appId>" -var client_secret="<password>"

Once you created all resources on AKS you can get the credentials so that you can use kubectl to configure your cluster:

az aks get-credentials --resource-group agonesRG --name test-cluster

Check that you have access to the Kubernetes cluster:

kubectl get nodes

Configurable parameters:

log_level - possible values: Fatal, Error, Warn, Info, Debug (default is “info”)
cluster_name - the name of the AKS cluster (default is “agones-terraform-example”)
agones_version - the version of agones to install (an empty string, which is the default, is the latest version from the Helm repository)
machine_type - node machine type for hosting game servers (default is “Standard_D2_v2”)
disk_size - disk size of the node
region - the location of the cluster
node_count - count of game server nodes for the default node pool (default is “4”)
feature_gates - a list of alpha and beta version features to enable. For example, “PlayerTracking=true&ContainerPortAllocation=true”
gameserver_minPort - the lower bound of the port range which gameservers will listen on (default is “7000”)
gameserver_maxPort - the upper bound of the port range which gameservers will listen on (default is “8000”)
gameserver_namespaces - a list of namespaces which will be used to run gameservers (default is ["default"]). For example ["default", "xbox-gameservers", "mobile-gameservers"]
force_update - whether or not to force the replacement/update of resource (default is true, false may be required to prevent immutability errors when updating the configuration)

Uninstall the Agones and delete AKS cluster

Run next command to delete all Terraform provisioned resources:

terraform destroy

Reference

Details on how you can authenticate your AKS terraform provider using official instructions.

Next Steps

Confirm Agones is up and running

Docs: Install Agones using Helm

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Helm package manager 3.2.3+
Supported Kubernetes Cluster

Helm 3

Installing the Chart

To install the chart with the release name my-release using our stable helm repository:

helm repo add agones https://agones.dev/chart/stable
helm repo update
helm install my-release --namespace agones-system --create-namespace agones/agones

We recommend installing Agones in its own namespaces, such as agones-system as shown above. If you want to use a different namespace, you can use the helm --namespace parameter to specify.

When running in production, Agones should be scheduled on a dedicated pool of nodes, distinct from where Game Servers are scheduled for better isolation and resiliency. By default Agones prefers to be scheduled on nodes labeled with agones.dev/agones-system=true and tolerates node taint agones.dev/agones-system=true:NoExecute. If no dedicated nodes are available, Agones will run on regular nodes, but that’s not recommended for production use. For instructions on setting up a dedicated node pool for Agones, see the Agones installation instructions for your preferred environment.

The command deploys Agones on the Kubernetes cluster with the default configuration. The configuration section lists the parameters that can be configured during installation.

Tip

List all releases using helm list --all-namespaces

Namespaces

By default Agones is configured to work with game servers deployed in the default namespace. If you are planning to use another namespace you can configure Agones via the parameter gameservers.namespaces.

For example to use default and xbox namespaces:

kubectl create namespace xbox
helm install my-release agones/agones --set "gameservers.namespaces={default,xbox}" --namespace agones-system

Note

You need to create your namespaces before installing Agones.

If you want to add a new namespace afterward upgrade your release:

kubectl create namespace ps4
helm upgrade my-release agones/agones --reuse-values --set "gameservers.namespaces={default,xbox,ps4}" --namespace agones-system

Uninstalling the Chart

To uninstall/delete the my-release deployment:

helm uninstall my-release --namespace=agones-system

RBAC

By default, agones.rbacEnabled is set to true. This enables RBAC support in Agones and must be true if RBAC is enabled in your cluster.

The chart will take care of creating the required service accounts and roles for Agones.

If you have RBAC disabled, or to put it another way, ABAC enabled, you should set this value to false.

Configuration

The following tables lists the configurable parameters of the Agones chart and their default values.

General

Parameter	Description	Default
`agones.featureGates`	A URL query encoded string of Flags to enable/disable e.g. `Example=true&OtherThing=false`. Any value accepted by strconv.ParseBool(string) can be used as a boolean value	``
`agones.rbacEnabled`	Creates RBAC resources. Must be set for any cluster configured with RBAC	`true`
`agones.registerWebhooks`	Registers the webhooks used for the admission controller	`true`
`agones.registerApiService`	Registers the apiservice(s) used for the Kubernetes API extension	`true`
`agones.registerServiceAccounts`	Attempts to create service accounts for the controllers	`true`
`agones.createPriorityClass`	Attempts to create priority classes for the controllers	`true`
`agones.priorityClassName`	Name of the priority classes to create	`agones-system`
`agones.requireDedicatedNodes`	Forces Agones system components to be scheduled on dedicated nodes, only applies to the GKE Standard without node auto-provisioning	`false`

Custom Resource Definitions

Parameter	Description	Default
`agones.crds.install`	Install the CRDs with this chart. Useful to disable if you want to subchart (since crd-install hook is broken), so you can copy the CRDs into your own chart.	`true`
`agones.crds.cleanupOnDelete`	Run the pre-delete hook to delete all GameServers and their backing Pods when deleting the helm chart, so that all CRDs can be removed on chart deletion	`true`
`agones.crds.cleanupJobTTL`	The number of seconds for Kubernetes to delete the associated Job and Pods of the pre-delete hook after it completes, regardless if the Job is successful or not. Set to `0` to disable cleaning up the Job or the associated Pods.	`60`

Metrics

Parameter	Description	Default
`agones.metrics.prometheusServiceDiscovery`	Adds annotations for Prometheus ServiceDiscovery (and also Strackdriver)	`true`
`agones.metrics.prometheusEnabled`	Enables controller metrics on port `8080` and path `/metrics`	`true`
`agones.metrics.stackdriverEnabled`	Enables Stackdriver exporter of controller metrics	`false`
`agones.metrics.stackdriverProjectID`	This overrides the default gcp project id for use with stackdriver	``
`agones.metrics.stackdriverLabels`	A set of default labels to add to all stackdriver metrics generated in form of key value pair (`key=value,key2=value2`). By default metadata are automatically added using Kubernetes API and GCP metadata enpoint.	``
`agones.metrics.serviceMonitor.interval`	Default scraping interval for ServiceMonitor	`30s`

Service Accounts

Parameter	Description	Default
`agones.serviceaccount.controller.name`	Service account name for the controller	`agones-controller`
`agones.serviceaccount.controller.annotations`	Annotations added to the Agones controller service account	`{}`
`agones.serviceaccount.sdk.name`	Service account name for the sdk	`agones-sdk`
`agones.serviceaccount.sdk.annotations`	A map of namespaces to maps of Annotations added to the Agones SDK service account for the specified namespaces	`{}`
`agones.serviceaccount.allocator.name`	Service account name for the allocator	`agones-allocator`
`agones.serviceaccount.allocator.annotations`	Annotations added to the Agones allocator service account	`{}`

Container Images

Parameter	Description	Default
`agones.image.registry`	Global image registry for all the Agones system images	`us-docker.pkg.dev/agones-images/release`
`agones.image.tag`	Global image tag for all images	`1.40.0`
`agones.image.controller.name`	Image name for the controller	`agones-controller`
`agones.image.controller.pullPolicy`	Image pull policy for the controller	`IfNotPresent`
`agones.image.controller.pullSecret`	Image pull secret for the controller, allocator, sdk and ping image. Should be created both in `agones-system` and `default` namespaces	``
`agones.image.sdk.name`	Image name for the sdk	`agones-sdk`
`agones.image.sdk.tag`	Image tag for the sdk	value of `agones.image.tag`
`agones.image.sdk.cpuRequest`	The cpu request for sdk server container	`30m`
`agones.image.sdk.cpuLimit`	The cpu limit for the sdk server container	`0` (none)
`agones.image.sdk.memoryRequest`	The memory request for sdk server container	`0` (none)
`agones.image.sdk.memoryLimit`	The memory limit for the sdk server container	`0` (none)
`agones.image.sdk.alwaysPull`	Tells if the sdk image should always be pulled	`false`
`agones.image.ping.name`	Image name for the ping service	`agones-ping`
`agones.image.ping.tag`	Image tag for the ping service	value of `agones.image.tag`
`agones.image.ping.pullPolicy`	Image pull policy for the ping service	`IfNotPresent`
`agones.image.extensions.name`	Image name for extensions	`agones-extensions`
`agones.image.extensions.pullPolicy`	Image pull policy for extensions	`IfNotPresent`

Agones Controller

Parameter	Description	Default
`agones.controller.replicas`	The number of replicas to run in the `agones-controller` deployment.	`2`
`agones.controller.pdb.minAvailable`	Description of the number of pods from that set that must still be available after the eviction, even in the absence of the evicted pod. Can be either an absolute number or a percentage. Mutually Exclusive with `maxUnavailable`	`1`
`agones.controller.pdb.maxUnavailable`	Description of the number of pods from that set that can be unavailable after the eviction. It can be either an absolute number or a percentage Mutually Exclusive with `minAvailable`	``
`agones.controller.http.port`	Port to use for liveness probe service and metrics	`8080`
`agones.controller.healthCheck.initialDelaySeconds`	Initial delay before performing the first probe (in seconds)	`3`
`agones.controller.healthCheck.periodSeconds`	Seconds between every liveness probe (in seconds)	`3`
`agones.controller.healthCheck.failureThreshold`	Number of times before giving up (in seconds)	`3`
`agones.controller.healthCheck.timeoutSeconds`	Number of seconds after which the probe times out (in seconds)	`1`
`agones.controller.resources`	Controller resource requests/limit	`{}`
`agones.controller.generateTLS`	Set to true to generate TLS certificates or false to provide your own certificates	`true`
`agones.controller.tlsCert`	Custom TLS certificate provided as a string	``
`agones.controller.tlsKey`	Custom TLS private key provided as a string	``
`agones.controller.nodeSelector`	Controller node labels for pod assignment	`{}`
`agones.controller.tolerations`	Controller toleration labels for pod assignment	`[]`
`agones.controller.affinity`	Controller affinity settings for pod assignment	`{}`
`agones.controller.annotations`	Annotations added to the Agones controller pods	`{}`
`agones.controller.numWorkers`	Number of workers to spin per resource type	`100`
`agones.controller.apiServerQPS`	Maximum sustained queries per second that controller should be making against API Server	`400`
`agones.controller.apiServerQPSBurst`	Maximum burst queries per second that controller should be making against API Server	`500`
`agones.controller.logLevel`	Agones Controller Log level. Log only entries with that severity and above	`info`
`agones.controller.persistentLogs`	Store Agones controller logs in a temporary volume attached to a container for debugging	`true`
`agones.controller.persistentLogsSizeLimitMB`	Maximum total size of all Agones container logs in MB	`10000`
`agones.controller.disableSecret`	Disables the creation of any allocator secrets. If true, you MUST provide the `{agones.releaseName}-cert` secrets before installation.	`false`
`agones.controller.customCertSecretPath`	Remap cert-manager path to server.crt and server.key	`{}`
`agones.controller.allocationApiService.annotations`	Annotations added to the Agones apiregistration	`{}`
`agones.controller.allocationApiService.disableCaBundle`	Disable ca-bundle so it can be injected by cert-manager	`false`
`agones.controller.validatingWebhook.annotations`	Annotations added to the Agones validating webhook	`{}`
`agones.controller.validatingWebhook.disableCaBundle`	Disable ca-bundle so it can be injected by cert-manager	`false`
`agones.controller.mutatingWebhook.annotations`	Annotations added to the Agones mutating webhook	`{}`
`agones.controller.mutatingWebhook.disableCaBundle`	Disable ca-bundle so it can be injected by cert-manager	`false`
`agones.controller.allocationBatchWaitTime`	Wait time between each allocation batch when performing allocations in controller mode	`500ms`
`agones.controller.topologySpreadConstraints`	Ensures better resource utilization and high availability by evenly distributing Pods in the agones-system namespace	`{}`

Ping Service

Parameter	Description	Default
`agones.ping.install`	Whether to install the ping service	`true`
`agones.ping.replicas`	The number of replicas to run in the deployment	`2`
`agones.ping.http.expose`	Expose the http ping service via a Service	`true`
`agones.ping.http.response`	The string response returned from the http service	`ok`
`agones.ping.http.port`	The port to expose on the service	`80`
`agones.ping.http.serviceType`	The Service Type of the HTTP Service	`LoadBalancer`
`agones.ping.http.nodePort`	Static node port to use for HTTP ping service. (Only applies when `agones.ping.http.serviceType` is `NodePort`.)	`0`
`agones.ping.http.loadBalancerIP`	The Load Balancer IP of the HTTP Service load balancer. Only works if the Kubernetes provider supports this option.	``
`agones.ping.http.loadBalancerSourceRanges`	The Load Balancer SourceRanges of the HTTP Service load balancer. Only works if the Kubernetes provider supports this option.	`[]`
`agones.ping.http.annotations`	Annotations added to the Agones ping http service	`{}`
`agones.ping.udp.expose`	Expose the udp ping service via a Service	`true`
`agones.ping.udp.rateLimit`	Number of UDP packets the ping service handles per instance, per second, per sender	`20`
`agones.ping.udp.port`	The port to expose on the service	`80`
`agones.ping.udp.serviceType`	The Service Type of the UDP Service	`LoadBalancer`
`agones.ping.udp.nodePort`	Static node port to use for UDP ping service. (Only applies when `agones.ping.udp.serviceType` is `NodePort`.)	`0`
`agones.ping.udp.loadBalancerIP`	The Load Balancer IP of the UDP Service load balancer. Only works if the Kubernetes provider supports this option.	``
`agones.ping.udp.loadBalancerSourceRanges`	The Load Balancer SourceRanges of the UDP Service load balancer. Only works if the Kubernetes provider supports this option.	`[]`
`agones.ping.udp.annotations`	Annotations added to the Agones ping udp service	`{}`
`agones.ping.healthCheck.initialDelaySeconds`	Initial delay before performing the first probe (in seconds)	`3`
`agones.ping.healthCheck.periodSeconds`	Seconds between every liveness probe (in seconds)	`3`
`agones.ping.healthCheck.failureThreshold`	Number of times before giving up (in seconds)	`3`
`agones.ping.healthCheck.timeoutSeconds`	Number of seconds after which the probe times out (in seconds)	`1`
`agones.ping.resources`	Ping pods resource requests/limit	`{}`
`agones.ping.nodeSelector`	Ping node labels for pod assignment	`{}`
`agones.ping.tolerations`	Ping toleration labels for pod assignment	`[]`
`agones.ping.affinity`	Ping affinity settings for pod assignment	`{}`
`agones.ping.annotations`	Annotations added to the Agones ping pods	`{}`
`agones.ping.updateStrategy`	The strategy to apply to the allocator deployment	`{}`
`agones.ping.pdb.enabled`	Set to `true` to enable the creation of a PodDisruptionBudget for the ping deployment	`false`
`agones.ping.pdb.minAvailable`	Description of the number of pods from that set that must still be available after the eviction, even in the absence of the evicted pod. Can be either an absolute number or a percentage. Mutually Exclusive with `maxUnavailable`	`1`
`agones.ping.pdb.maxUnavailable`	Description of the number of pods from that set that can be unavailable after the eviction. It can be either an absolute number or a percentage Mutually Exclusive with `minAvailable`	``
`agones.ping.topologySpreadConstraints`	Ensures better resource utilization and high availability by evenly distributing Pods in the agones-system namespace	`{}`

Allocator Service

Parameter	Description	Default
`agones.allocator.apiServerQPS`	Maximum sustained queries per second that an allocator should be making against API Server	`400`
`agones.allocator.apiServerQPSBurst`	Maximum burst queries per second that an allocator should be making against API Server	`500`
`agones.allocator.remoteAllocationTimeout`	Remote allocation call timeout.	`10s`
`agones.allocator.totalRemoteAllocationTimeout`	Total remote allocation timeout including retries.	`30s`
`agones.allocator.logLevel`	Agones Allocator Log level. Log only entries with that severity and above	`info`
`agones.allocator.install`	Whether to install the allocator service	`true`
`agones.allocator.replicas`	The number of replicas to run in the deployment	`3`
`agones.allocator.service.name`	Service name for the allocator	`agones-allocator`
`agones.allocator.service.serviceType`	The Service Type of the HTTP Service	`LoadBalancer`
`agones.allocator.service.clusterIP`	The Cluster IP of the Agones allocator. If you want Headless Service for Agones Allocator, you can set `None` to clusterIP.	``
`agones.allocator.service.loadBalancerIP`	The Load Balancer IP of the Agones allocator load balancer. Only works if the Kubernetes provider supports this option.	``
`agones.allocator.service.loadBalancerSourceRanges`	The Load Balancer SourceRanges of the Agones allocator load balancer. Only works if the Kubernetes provider supports this option.	`[]`
`agones.allocator.service.annotations`	Annotations added to the Agones allocator service	`{}`
`agones.allocator.service.http.enabled`	If true the allocator service will respond to REST requests	`true`
`agones.allocator.service.http.appProtocol`	The `appProtocol` to set on the Service for the http allocation port. If left blank, no value is set.	``
`agones.allocator.service.http.port`	The port that is exposed externally by the allocator service for REST requests	`443`
`agones.allocator.service.http.portName`	The name of exposed port	`http`
`agones.allocator.service.http.targetPort`	The port that is used by the allocator pod to listen for REST requests. Note that the allocator server cannot bind to low numbered ports.	`8443`
`agones.allocator.service.http.nodePort`	If the ServiceType is set to “NodePort”, this is the NodePort that the allocator http service is exposed on.	`30000-32767`
`agones.allocator.service.grpc.enabled`	If true the allocator service will respond to gRPC requests	`true`
`agones.allocator.service.grpc.port`	The port that is exposed externally by the allocator service for gRPC requests	`443`
`agones.allocator.service.grpc.portName`	The name of exposed port	``
`agones.allocator.service.grpc.appProtocol`	The `appProtocol` to set on the Service for the gRPC allocation port. If left blank, no value is set.	``
`agones.allocator.service.grpc.nodePort`	If the ServiceType is set to “NodePort”, this is the NodePort that the allocator gRPC service is exposed on.	`30000-32767`
`agones.allocator.service.grpc.targetPort`	The port that is used by the allocator pod to listen for gRPC requests. Note that the allocator server cannot bind to low numbered ports.	`8443`
`agones.allocator.generateClientTLS`	Set to true to generate client TLS certificates or false to provide certificates in `certs/allocator/allocator-client.default/*`	`true`
`agones.allocator.generateTLS`	Set to true to generate TLS certificates or false to provide your own certificates	`true`
`agones.allocator.disableMTLS`	Turns off client cert authentication for incoming connections to the allocator.	`false`
`agones.allocator.disableTLS`	Turns off TLS security for incoming connections to the allocator.	`false`
`agones.allocator.disableSecretCreation`	Disables the creation of any allocator secrets. If true, you MUST provide the `allocator-tls`, `allocator-tls-ca`, and `allocator-client-ca` secrets before installation.	`false`
`agones.allocator.tlsCert`	Custom TLS certificate provided as a string	``
`agones.allocator.tlsKey`	Custom TLS private key provided as a string	``
`agones.allocator.clientCAs`	A map of secret key names to allowed client CA certificates provided as strings	`{}`
`agones.allocator.tolerations`	Allocator toleration labels for pod assignment	`[]`
`agones.allocator.affinity`	Allocator affinity settings for pod assignment	`{}`
`agones.allocator.annotations`	Annotations added to the Agones allocator pods	`{}`
`agones.allocator.resources`	Allocator pods resource requests/limit	`{}`
`agones.allocator.labels`	Labels Added to the Agones Allocator pods	`{}`
`agones.allocator.readiness.initialDelaySeconds`	Initial delay before performing the first probe (in seconds)	`3`
`agones.allocator.readiness.periodSeconds`	Seconds between every liveness probe (in seconds)	`3`
`agones.allocator.readiness.failureThreshold`	Number of times before giving up (in seconds)	`3`
`agones.allocator.nodeSelector`	Allocator node labels for pod assignment	`{}`
`agones.allocator.serviceMetrics.name`	Second Service name for the allocator	`agones-allocator-metrics-service`
`agones.allocator.serviceMetrics.annotations`	Annotations added to the Agones allocator second Service	`{}`
`agones.allocator.serviceMetrics.http.port`	The port that is exposed within cluster by the allocator service for http requests	`8080`
`agones.allocator.serviceMetrics.http.portName`	The name of exposed port	`http`
`agones.allocator.allocationBatchWaitTime`	Wait time between each allocation batch when performing allocations in allocator mode	`500ms`
`agones.allocator.updateStrategy`	The strategy to apply to the ping deployment	`{}`
`agones.allocator.pdb.enabled`	Set to `true` to enable the creation of a PodDisruptionBudget for the allocator deployment	`false`
`agones.allocator.pdb.minAvailable`	Description of the number of pods from that set that must still be available after the eviction, even in the absence of the evicted pod. Can be either an absolute number or a percentage. Mutually Exclusive with `maxUnavailable`	`1`
`agones.allocator.pdb.maxUnavailable`	Description of the number of pods from that set that can be unavailable after the eviction. It can be either an absolute number or a percentage. Mutually Exclusive with `minAvailable`	``
`agones.allocator.topologySpreadConstraints`	Ensures better resource utilization and high availability by evenly distributing Pods in the agones-system namespace	`{}`

Extensions

Parameter	Description	Default
`agones.extensions.http.port`	Port to use for liveness probe service and metrics	`8080`
`agones.extensions.healthCheck.initialDelaySeconds`	Initial delay before performing the first probe (in seconds)	`3`
`agones.extensions.healthCheck.periodSeconds`	Seconds between every liveness probe (in seconds)	`3`
`agones.extensions.healthCheck.failureThreshold`	Number of times before giving up (in seconds)	`3`
`agones.extensions.healthCheck.timeoutSeconds`	Number of seconds after which the probe times out (in seconds)	`1`
`agones.extensions.resources`	Extensions resource requests/limit	`{}`
`agones.extensions.generateTLS`	Set to true to generate TLS certificates or false to provide your own certificates	`true`
`agones.extensions.tlsCert`	Custom TLS certificate provided as a string	``
`agones.extensions.tlsKey`	Custom TLS private key provided as a string	``
`agones.extensions.nodeSelector`	Extensions node labels for pod assignment	`{}`
`agones.extensions.tolerations`	Extensions toleration labels for pod assignment	`[]`
`agones.extensions.affinity`	Extensions affinity settings for pod assignment	`{}`
`agones.extensions.annotations`	Annotations added to the Agones extensions pods	`{}`
`agones.extensions.numWorkers`	Number of workers to spin per resource type	`100`
`agones.extensions.apiServerQPS`	Maximum sustained queries per second that extensions should be making against API Server	`400`
`agones.extensions.apiServerQPSBurst`	Maximum burst queries per second that extensions should be making against API Server	`500`
`agones.extensions.logLevel`	Agones Extensions Log level. Log only entries with that severity and above	`info`
`agones.extensions.persistentLogs`	Store Agones extensions logs in a temporary volume attached to a container for debugging	`true`
`agones.extensions.persistentLogsSizeLimitMB`	Maximum total size of all Agones container logs in MB	`10000`
`agones.extensions.disableSecret`	Disables the creation of any allocator secrets. If true, you MUST provide the `{agones.releaseName}-cert` secrets before installation.	`false`
`agones.extensions.customCertSecretPath`	Remap cert-manager path to server.crt and server.key	`{}`
`agones.extensions.allocationApiService.annotations`	Annotations added to the Agones apiregistration	`{}`
`agones.extensions.allocationApiService.disableCaBundle`	Disable ca-bundle so it can be injected by cert-manager	`false`
`agones.extensions.validatingWebhook.annotations`	Annotations added to the Agones validating webhook	`{}`
`agones.extensions.validatingWebhook.disableCaBundle`	Disable ca-bundle so it can be injected by cert-manager	`false`
`agones.extensions.mutatingWebhook.annotations`	Annotations added to the Agones mutating webhook	`{}`
`agones.extensions.mutatingWebhook.disableCaBundle`	Disable ca-bundle so it can be injected by cert-manager	`false`
`agones.extensions.allocationBatchWaitTime`	Wait time between each allocation batch when performing allocations in controller mode	`500ms`
`agones.extensions.pdb.minAvailable`	Description of the number of pods from that set that must still be available after the eviction, even in the absence of the evicted pod. Can be either an absolute number or a percentage. Mutually Exclusive with maxUnavailable	`1`
`agones.extensions.pdb.maxUnavailable`	Description of the number of pods from that set that can be unavailable after the eviction. It can be either an absolute number or a percentage Mutually Exclusive with `minAvailable`	``
`agones.extensions.replicas`	The number of replicas to run in the deployment	`2`
`agones.extensions.topologySpreadConstraints`	Ensures better resource utilization and high availability by evenly distributing Pods in the agones-system namespace	`{}`

GameServers

Helm Installation

Parameter	Description	Default
`helm.installTests`	Add an ability to run `helm test agones` to verify the installation	`false`

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example,

helm install my-release --namespace agones-system \
  --set gameservers.minPort=1000,gameservers.maxPort=5000 agones

The above command will deploy Agones controllers to agones-system namespace. Additionally Agones will use a dynamic GameServers’ port allocation range of 1000-5000.

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,

helm install my-release --namespace agones-system -f values.yaml agones/agones

Tip

You can use the default values.yaml

Helm test

This test would create a GameServer resource and delete it afterwards.

Tip

In order to use helm test command described in this section you need to set helm.installTests helm parameter to true.

Check the Agones installation by running the following command:

helm test my-release -n agones-system

You should see a successful output similar to this :

NAME: my-release
LAST DEPLOYED: Wed Mar 29 06:13:23 2023
NAMESPACE: agones-system
STATUS: deployed
REVISION: 4
TEST SUITE:     my-release-test
Last Started:   Wed Mar 29 06:17:52 2023
Last Completed: Wed Mar 29 06:18:10 2023
Phase:          Succeeded

Controller TLS Certificates

By default agones chart generates tls certificates used by the admission controller, while this is handy, it requires the agones controller to restart on each helm upgrade command.

Manual

For most use cases the controller would have required a restart anyway (eg: controller image updated). However if you really need to avoid restarts we suggest that you turn off tls automatic generation (agones.controller.generateTLS to false) and provide your own certificates (certs/server.crt,certs/server.key).

Tip

You can use our script located at cert.sh to generate them.

Cert-Manager

Another approach is to use cert-manager.io solution for cluster level certificate management.

In order to use the cert-manager solution, first install cert-manager on the cluster. Then, configure an Issuer/ClusterIssuer resource and last configure a Certificate resource to manage controller Secret. Make sure to configure the Certificate based on your system’s requirements, including the validity duration.

Here is an example of using a self-signed ClusterIssuer for configuring controller Secret where secret name is my-release-cert or {{ template "agones.fullname" . }}-cert:

#!/bin/bash
# Create a self-signed ClusterIssuer
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned
spec:
  selfSigned: {}
EOF

# Create a Certificate with IP for the my-release-cert )
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-release-agones-cert
  namespace: agones-system
spec:
  dnsNames:
    - agones-controller-service.agones-system.svc
  secretName: my-release-agones-cert
  issuerRef:
    name: selfsigned
    kind: ClusterIssuer
EOF

After the certificates are generated, we will want to inject caBundle into the controller and extensions webhook and disable the controller and extensions secret creation through the following values.yaml file.:

agones:
  controller:
    disableSecret: true
    customCertSecretPath:
    - key: ca.crt
      path: ca.crt
    - key: tls.crt
      path: server.crt
    - key: tls.key
      path: server.key
    allocationApiService:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    validatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    mutatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
  extensions:
    disableSecret: true
    customCertSecretPath:
    - key: ca.crt
      path: ca.crt
    - key: tls.crt
      path: server.crt
    - key: tls.key
      path: server.key
    allocationApiService:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    validatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    mutatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true

After copying the above yaml into a values.yaml file, use below command to install Agones:

helm install my-release --namespace agones-system --create-namespace --values values.yaml agones/agones

Reserved Allocator Load Balancer IP

In order to reuse the existing load balancer IP on upgrade or install the agones-allocator service as a LoadBalancer using a reserved static IP, a user can specify the load balancer’s IP with the agones.allocator.http.loadBalancerIP helm configuration parameter value. By setting the loadBalancerIP value:

The LoadBalancer is created with the specified IP, if supported by the cloud provider.
A self-signed server TLS certificate is generated for the IP, used by the agones-allocator service.

Next Steps

Confirm Agones is up and running

Docs: GameServer Counters and Lists

Mon, 08 Jan 2024 00:00:00 +0000

Warning

The Counters and Lists feature is currently Alpha, not enabled by default, and may change in the future.

Use the FeatureGate CountsAndLists to enable and test this feature.

See the Feature Gate documentation for details on how to enable features.

Counters and Lists is provided as a way to track arbitrary integer counter values as well as lists of values against a GameServer by a user provided key.

Combined with the ability to set and manipulate max capacity values at runtime for each counter and list, allows Agones to also provide Allocation, Fleet scheduling and Fleet autoscaling based on this functionality, such that it supports a wide variety of use cases, including, but not limited to:

Connected player listing tracking, Allocation filtering and autoscaling based on available capacity.
Multi-tenant server room counting, Allocation filtering and autoscaling based on available capacity.
Game specific GameServer weighting on Allocation.
…any other use case that requires a list of values or a counter aligned with a GameServer for tracking purposes.

Declaration

All keys for either Counters and Lists must be declared at creation time within the GameServerSpec before being utilised, and keys cannot be added or deleted from GameServers past their initial creation.

For example, if we want to use a Counter of rooms to track the number of game session rooms that exist in this GameServer instance, while also tracking a List of currently connected players, we could implement the following:

apiVersion: agones.dev/v1
kind: Fleet
metadata:
  name: simple-game-server
spec:
  replicas: 2
  template:
    spec:
      ports:
        - name: default
          containerPort: 7654
      counters:
        rooms: # room counter
          count: 0
          capacity: 10
      lists:
        players: # players list
          values: []
      template:
        spec:
          containers:
            - name: simple-game-server
              image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32

Both Counters and Lists can have a capacity value, which indicated the maximum counter value, or number of items that can be stored in the list.

In the above example, the room Counter has a capacity of 10, which means that the count cannot go past that value.

See the GameServer reference for all configurable options.

Retrieval

We now have several ways to retrieve this Counter and List information from a GameServer, depending on your use case.

Kubernetes API

If you wish to retrieve or view current Counter or List values from outside your GameServer, you are able to do this through the Kubernetes API, or similarly through kubectl.

Counter values and capacities for are stored on a GameServer resource instance under GameServer.Status.Counters by key, and GameServer.Status.Lists stores List value arrays and capacities by key as well.

Therefore, in the above examples, the GameServer.Status.Counters[rooms].Count and GameServer.Status.Counters[rooms].Capacity would have the current Counter value and capacity for the room counters.

Subsequently GameServer.Status.Lists[players].Values stores the array of values for the list and GameServer.Status.Lists[players].Capacity is the current capacity for the player tracking List.

Check the API reference for GameServerStatus for all the details on the data structure.

SDK

Counter and Lists values can be accessed through the Agones SDK when that information is required within your game server process.

For example, to retrieve the above room counter, each language SDK has some implementation of the SDK.Alpha().GetCounterCount("room") function that returns the current value of the counter. Similarly, to retrieve the players list, we can use the SDK.Alpha().GetListValues("players") function.

The special ability of the SDK retrieval operations, is that is also tracks any modifications that have been made through the SDK, that have yet to be persisted to the GameServer, and will return values with that data included.

This means that any modifications made to Counters and Lists sent through the SDK can be immediately and atomically retrieved from the SDK from the game server binary with accurate information without having to wait for it be persisted to the GameServer.

See the SDK Guide for the full set of data retrieval functions that are available.

Manipulation

We also have several ways to manipulate the Counter and List information stored on a GameServer instance, depending on your use case.

SDK

Counter and Lists values can be modified through the Agones SDK when you wish to be able to edit that information within your game server process.

For example, to increment the above room counter by 1, each language SDK has some implementation of the SDK.Alpha().IncrementCounter("room", 1) function thatincrements the counter by 1. Similarly, to add the value player1 to the players list, we can use the SDK.Alpha().AppendListValue("players", "player1") function.

See the SDK Guide for the full set of modification functions that are available.

Allocation

When performing a GameServer allocation, you may want to manipulate the Counter and/or List information atomically on a GameServer allocation operation.

For example, you may wish to increment the room counter to indicate a new game session has started on the returned GameServer instance, or provide the connecting player information to the player list to make that available to the game server binary before the player connects.

This can be done through GameServerAllocation.Spec.Counters and GameServerAllocation.Spec.Lists, which provide CounterAction and ListAction configuration respectively.

For example, if on allocation we wished to increment the room counter by 1 and add player1 to the players list, this could be done with the following GameServerAllocation specification:

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        agones.dev/fleet: simple-game-server
  counters:
    rooms:
      action: Increment
      amount: 1
  lists:
    players:
      addValues:
        - player1

Counter and List changes made through the GameServerAllocation functionality, will be eventually consistent in their availability via the SDK retrieval functions for performance reasons.

Performing these data changes as part of the Allocation also does not guarantee that the counter or list is past its capacity value. In the event that the capacity is exceeded, the allocation will succeed, but the Counter or List values will not be updated. If you want to ensure there is capacity available for the Allocation operation, see Allocation Filtering and Prioritisation below.

See the Allocation Reference for all the Allocation configuration options.

Kubernetes API

Counter values and capacities are stored on a GameServer resource instance under GameServer.Status.Counters by key, and GameServer.Status.Lists stores List value arrays and capacities by key as well. Therefore, they can be modified through the Kubernetes API either through Kubernetes client libraries or manually through kubectl edit.

Counter and List changes made through the GameServerAllocation functionality, will be eventually consistent in their availability via the SDK retrieval functions for performance reasons.

Check out the Access Agones via the Kubernetes API guide for a more in depth guide on how to interact with Agones resources through the Kubernetes API programmatically.

Allocation Filtering and Prioritisation

Counters and Lists can also be used as filtering properties when performing an allocation through either the GameServerAllocation.Spec.Selectors.Counts or GameServerAllocation.Spec.Selectors.Lists properties.

If we want to expand the above example to ensure there is always room for the room Counter increment and room for the player List to add player1, we can add the following to the GameServerAllocation specification:

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        agones.dev/fleet: simple-game-server
      counters:
        rooms:
          minAvailable: 1
      lists:
        players:
          minAvailable: 1
  counters:
    rooms:
      action: Increment
      amount: 1
  lists:
    players:
      addValues:
        - player1

Only GameServer instances that have a capacity value of at least one more than the Counter value, or List length will be included in the potential allocation result - thereby ensuring that both room and player operations will succeed, assuming a GameServer is found.

This can be combined with the ability to define multiple selectors to create quite sophisticated GameServer selection options that can attempt to find the most appropriate GameServer.

See the Allocation Reference for more details.

The priorities block also gives you options to prioritise specific GameServers over others when performing an Allocation, depending on the Allocation Strategy.

Packed: The usual infrastructure optimisation strategy still applies, but the priorities block is used as a tie-breaker within the least utilised infrastructure, to ensure optimal infrastructure usage while also allowing some custom prioritisation of GameServers.
Distributed: The entire selection of GameServers will be sorted by this priority list to provide the order that GameServers will be allocated by.

For example, if we wanted to select a GameServer that had:

At least one room available in the GameServer capacity.
First check Allocated GameServers and falling back to Ready GameServers if there aren’t any.
Include general infrastructure optimisation for least usage.
And also ensure the GameServers that were most full are allocated first (after infrastructure optimisations where applied).

We could implement the following GameServerAllocation:

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
spec:
  scheduling: Packed
  # Choose most full `GameServers` first
  priorities:
    - type: Counter
      key: rooms
      order: Ascending
  selectors:
    # First check to see if we can back-fill an already allocated `GameServer`
    - gameServerState: Allocated
      matchLabels:
        agones.dev/fleet: simple-game-server
      counters:
        rooms:
          minAvailable: 1
    # If we can't, then go get a `Ready` `GameServer`.
    - gameServerState: Ready
      matchLabels:
        agones.dev/fleet: simple-game-server
      counters:
        rooms:
          minAvailable: 1
  counters:
    rooms:
      action: Increment
      amount: 1

For more details on how Agones implements infrastructure optimisation, see the documentation on Scheduling and Autoscaling.

Fleet Scale Down Prioritisation

Another optimisation control you can apply with Fleets when Agones’ scaled them down, is to also set priorities to influence the order in which GameServers are shutdown and deleted.

While neither players or rooms are particularly good examples for this functionality, if we wanted to ensure that Ready GameServers with the most available capacity rooms were a factor when scaling down a Fleet we could implement the following:

apiVersion: agones.dev/v1
kind: Fleet
metadata:
  name: simple-game-server
spec:
  replicas: 2
  priorities:
    - type: Counter
      key: rooms
      order: Descending
  template:
    spec:
      ports:
        - name: default
          containerPort: 7654
      counters:
        rooms: # room counter
          count: 0
          capacity: 10
      template:
        spec:
          containers:
            - name: simple-game-server
              image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32

Depending on the Fleet allocation strategy, the priorities block will influence scale down logic as follows:

Packed: The usual infrastructure optimisation strategy still applies, but the priorities block is used as a tie-breaker within the least utilised infrastructure, to ensure optimal infrastructure usage while also allowing some custom prioritisation of GameServers.
Distributed: The entire selection of GameServers will be sorted by this priority list to provide the order that GameServers will be scaled down by.

See Fleet Reference for all the configuration options.

Autoscaling

Counters and Lists expands on Fleet Autoscaling capabilities, by allowing you to autoscale based on available capacity across the Fleet as a unit, rather than the less granular unit of individual GameServers instances.

This means, we can implement an Autoscaling strategy of “always make sure there are 5 free rooms available for new players at all times” like so:

apiVersion: autoscaling.agones.dev/v1
kind: FleetAutoscaler
metadata:
  name: simple-game-server
spec:
  fleetName: fleet-example
  policy:
    type: Counter
    counter:
      key: rooms
      bufferSize: 5
      maxCapacity: 100

See the Fleet Autoscaling Reference for all the configuration options.

Metrics

Metrics are exported, using the key that the metric is stored under as a label on the metrics, in aggregate across all GameServers within a Fleet, exporting aggregate numeric totals for Counters and Lists as gauge metrics.

Name	Description	Type
agones_fleet_counters	Aggregate Metrics for Counters within a Fleet, including total capacity and count values	gauge
agones_fleet_lists	Aggregate Metrics for Lists within a Fleet, including total capacity and List lengths	gauge

See Metrics available for the full list of available metrics and how to use them.

Next Steps

Check out how to access Agones resource via the Kubernetes API.
Have a look at the external Allocator Service to make integrating Allocation into your workflow easier.

Docs: Canary Testing a new Fleet

Tue, 27 Jul 2021 23:49:14 +0000

To canary release/test a new Fleet, we can run a small, fixed size Fleet of the new version of our GameServer, while also running the current stable production version.

Allocations can then prefer to come from the canary Fleet, but if all GameServers are already allocated from the canary Fleet, players will be allocated to the current stable Fleets.

Over time, if the monitoring of those playing on the canary Fleet is working as expected, the size of the canary Fleet can be grown until you feel confident in its stability.

Once confidence has been achieved, the configuration for stable Fleet can be updated to match the canary (usually triggering a rolling update). The canary Fleet can then be deleted or updated to a new testing version of the game server process.

Sample `GameServerAllocation`

To ensure we don’t have to change the Allocation system every time we have a canary Fleet, in this example, we will state that in our system, the label canary: "true" will be added to any canary Fleet in the cluster.

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        canary: "true"
    - matchLabels:
        agones.dev/fleet: stable

The above Allocation will then preferentially choose the Fleet that has GameServers with the label and key value ofcanary:"true", if it exists, and has remaining Ready GameServers, and if not, will apply the Allocation to the Fleet named “stable”.

Next Steps

Read about different Fleet update options and strategies that are available.
Have a look at all the options available on a GameServerAllocation.
Review the Fleet reference.

Docs: Player Tracking

Tue, 19 May 2020 00:00:00 +0000

Counters and Lists will eventually replace the Alpha functionality of Player Tracking, which will subsequently be removed from Agones.

If you are currently using this Alpha feature, we would love for you to test (and ideally migrate to!) this new functionality to Counters and Lists to ensure it meet all your needs.

Warning

The Player Tracking feature is currently Alpha, not enabled by default, and may change in the future.

Use the FeatureGate PlayerTracking to enable and test this feature.

See the Feature Gate documentation for details on how to enable features.

Managing GameServer Capacities

To track your GameServer current player capacity, Agones gives you the ability to both set an initial capacity at GameServer creation, as well be able to change it during the lifecycle of the GameServer through the Agones SDK.

To set the initial capacity, you can do so via GameServer.Spec.Players.InitialCapacity like so:

apiVersion: "agones.dev/v1"
kind: GameServer
metadata:
  name: "gs-example"
spec:
  # ...
  players:
    # set this GameServer's initial player capacity to 10
    initialCapacity: 10

From there, if you need to change the capacity of the GameSever as gameplay is in progress, you can also do so via SDK.Alpha().SetPlayerCapacity(count)

The current player capacity is represented in GameServer.Status.Players.Capacity resource value.

We can see this in action, when we look at the Status section of a GameServer resource , wherein the capacity has been set to 20:

...
Status:
  Address:    14.81.195.72
  Node Name:  gke-test-cluster-default-6cd0ba67-1mps
  Players:
    Capacity:  20
    Count:     0
    Ids:       <nil>
  Ports:
    Name:          gameport
    Port:          7983
  Reserved Until:  <nil>
  State:           Ready

From the SDK, the game server binary can also retrieve the current player capacity via SDK.Alpha().GetPlayerCapacity().

Note

Changing the capacity value here has no impact on players actually connected to or trying to connect to your server, as that is not a responsibility of Agones.

This functionality is for tracking purposes only.

Connecting and Disconnecting Players

As players connect and disconnect from your game, the Player Tracking functions enable you to track which players are currently connected.

It assumed that each player that connects has a unique token that identifies them as a player.

When a player connects to the game server binary, calling SDK.Alpha().PlayerConnect(playerID) with the unique player token will register them as connected, and store their player id.

At disconnection time, call SDK.Alpha().PlayerDisconnect(playerID) , which will deregister them and remove their player id from the list.

Each of these playerIDs is stored on GameServer.Status.Players.IDs, and the current count of connected players can be seen in GameServer.Status.Players.Count.

You can see this in action below in the GameServer Status section, where there are 4 players connected:

...
Status:
  Address:    39.82.196.74
  Node Name:  gke-test-cluster-default-6cd0ba77-1mps
  Players:
    Capacity:  10
    Count:     4
    Ids:
      xy8a
      m0ux
      71nj
      lpq5
  Ports:
    Name:          gameport
    Port:          7166
  Reserved Until:  <nil>
  State:           Ready

Note

Calling PlayerConnect or PlayerDisconnect functions will not connect or disconnect players, as that is not under the control of Agones.

This functionality is for tracking purposes only.

Checking Player Data

Not only is the connected player data stored on the GameServer resource, it is also stored in memory within the SDK, so that it can be used from within the game server binary as a realtime, thread safe, registry of connected players.

Therefore, if you want to:

Get the current player count, call SDK.Alpha().GetPlayerCount()
Check if a specific player is connected, call SDK.Alpha().IsPlayerConnected(playerID)
Retrieve the full list of connected players, call SDK.Alpha().GetConnectedPlayers()

Next Steps

Review the Player Tracking SDK Reference

Docs: Third Party Libraries and Tools

Fri, 15 May 2020 00:00:00 +0000

Client SDKs

Cubxity/AgonesKt - Agones Client SDK for Kotlin
AndreMicheletti/godot-agones-sdk - Agones Client SDK for Godot Engine
Infumia/agones4j - Agones Client SDK for Java

Messaging

Libraries or applications that implement messaging systems.

Octops/Agones Event Broadcaster - Broadcast Agones events to the external world
Octops/Agones Broadcaster HTTP - Expose Agones GameServers information via HTTP
Octops/Agones Relay HTTP - Publish Agones GameServers and Fleets details to HTTP endpoints

Controllers

Octops/Game Server Ingress Controller - Automatic Ingress configuration for Game Servers managed by Agones
Octops/Image Syncer - Watch Fleets and pre-pull images of game servers on every node running in the cluster
Octops/Fleet Garbage Collector - Delete Fleets based on its TTL

Allocation

agones-allocator-client - A client for testing allocation servers. Made by Fairwinds
Multi-cluster allocation demo - A demo project for multi-cluster allocation on Amazon EKS with Terraform templates.

Development Tools

Minikube Agones Cluster - Automates the creation of a complete Kubernetes/Agones Cluster locally, using Xonotic as a sample gameserver. Intended to provide a local environment for developers which approximates a production Agones deployment.

Docs: GameServerAllocation Specification

Sun, 07 Jul 2019 03:58:52 +0000

Allocation is the process of selecting the optimal GameServer that matches the filters defined in the GameServerAllocation specification below, and returning its details.

A successful Alloction moves the GameServer to the Allocated state, which indicates that it is currently active, likely with players on it, and should not be removed until SDK.Shutdown() is called, or it is explicitly manually deleted.

A full GameServerAllocation specification is available below and in the example folder for reference:

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  # GameServer selector from which to choose GameServers from.
  # Defaults to all GameServers.
  # matchLabels, matchExpressions, gameServerState and player filters can be used for filtering.
  # See: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more details on label selectors.
  # An ordered list of GameServer label selectors.
  # If the first selector is not matched, the selection attempts the second selector, and so on.
  # This is useful for things like smoke testing of new game servers.
  selectors:
    - matchLabels:
        agones.dev/fleet: green-fleet
        # [Stage:Alpha]
        # [FeatureFlag:PlayerAllocationFilter]
      players:
        minAvailable: 0
        maxAvailable: 99
    - matchLabels:
        agones.dev/fleet: blue-fleet
    - matchLabels:
        game: my-game
      matchExpressions:
        - {key: tier, operator: In, values: [cache]}
      # Specifies which State is the filter to be used when attempting to retrieve a GameServer
      # via Allocation. Defaults to "Ready". The only other option is "Allocated", which can be used in conjunction with
      # label/annotation/player selectors to retrieve an already Allocated GameServer.
      gameServerState: Ready
      # [Stage:Alpha]
      # [FeatureFlag:CountsAndLists]
      # counters: # selector for counter current values of a GameServer count
      #   rooms:
      #     minCount: 1 # minimum value. Defaults to 0.
      #     maxCount: 5 # maximum value. Defaults to max(int64)
      #     minAvailable: 1 # minimum available (current capacity - current count). Defaults to 0.
      #     maxAvailable: 10 # maximum available (current capacity - current count) Defaults to max(int64)
      # lists:
      #   players:
      #     containsValue: "x6k8z" # only match GameServers who has this value in the list. Defaults to "", which is all.
      #     minAvailable: 1 # minimum available (current capacity - current count). Defaults to 0.
      #     maxAvailable: 10 # maximum available (current capacity - current count) Defaults to 0, which translates to max(int64)
      # [Stage:Alpha]      
      # [FeatureFlag:PlayerAllocationFilter]
      # Provides a filter on minimum and maximum values for player capacity when retrieving a GameServer
      # through Allocation. Defaults to no limits.
      players:
        minAvailable: 0
        maxAvailable: 99
  # defines how GameServers are organised across the cluster.
  # Options include:
  # "Packed" (default) is aimed at dynamic Kubernetes clusters, such as cloud providers, wherein we want to bin pack
  # resources
  # "Distributed" is aimed at static Kubernetes clusters, wherein we want to distribute resources across the entire
  # cluster
  scheduling: Packed
  # Optional custom metadata that is added to the game server at allocation
  # You can use this to tell the server necessary session data
  metadata:
    labels:
      mode: deathmatch
    annotations:
      map:  garden22
  # [Stage: Alpha]
  # [FeatureFlag:CountsAndLists]
  # `Priorities` configuration alters the order in which `GameServers` are searched for matches to the configured `selectors`.
  #
  # Priority of sorting is in descending importance. I.e. The position 0 `priority` entry is checked first.
  #
  # For `Packed` strategy sorting, this priority list will be the tie-breaker within the least utilised infrastructure, to ensure optimal
  # infrastructure usage while also allowing some custom prioritisation of `GameServers`.
  #
  # For `Distributed` strategy sorting, the entire selection of `GameServers` will be sorted by this priority list to provide the
  # order that `GameServers` will be allocated by.
  # Optional.
  # priorities:
  # - type: Counter  # Whether a Counter or a List.
  #   key: rooms  # The name of the Counter or List.
  #   order: Ascending  # "Ascending" lists smaller available capacity first.
  # [Stage: Alpha]
  # [FeatureFlag:CountsAndLists]
  # Counter actions to perform during allocation. Optional.
  # counters:
  #   rooms:
  #     action: Increment # Either "Increment" or "Decrement" the Counter’s Count.
  #     amount: 1 # Amount is the amount to increment or decrement the Count. Must be a positive integer.
  #     capacity: 5 # Amount to update the maximum capacity of the Counter to this number. Min 0, Max int64.
  # List actions to perform during allocation. Optional.
  # lists:
  #   players:
  #     addValues: # appends values to a List’s Values array. Any duplicate values will be ignored
  #       - x7un
  #       - 8inz
  #     capacity: 40 # Updates the maximum capacity of the Counter to this number. Min 0, Max 1000.

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  # Deprecated, use field selectors instead.
  # GameServer selector from which to choose GameServers from.
  # Defaults to all GameServers.
  # matchLabels, matchExpressions, gameServerState and player filters can be used for filtering.
  # See: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more details on label selectors.
  # Deprecated, use field selectors instead.
  required:
    matchLabels:
      game: my-game
    matchExpressions:
      - {key: tier, operator: In, values: [cache]}
    # Specifies which State is the filter to be used when attempting to retrieve a GameServer
    # via Allocation. Defaults to "Ready". The only other option is "Allocated", which can be used in conjunction with
    # label/annotation/player selectors to retrieve an already Allocated GameServer.
    gameServerState: Ready
    # [Stage:Alpha]
    # [FeatureFlag:PlayerAllocationFilter]
    # Provides a filter on minimum and maximum values for player capacity when retrieving a GameServer
    # through Allocation. Defaults to no limits.
    players:
      minAvailable: 0
      maxAvailable: 99
  # Deprecated, use field selectors instead.
  # An ordered list of preferred GameServer label selectors
  # that are optional to be fulfilled, but will be searched before the `required` selector.
  # If the first selector is not matched, the selection attempts the second selector, and so on.
  # If any of the preferred selectors are matched, the required selector is not considered.
  # This is useful for things like smoke testing of new game servers.
  # This also support matchExpressions, gameServerState and player filters.
  preferred:
    - matchLabels:
        agones.dev/fleet: green-fleet
      # [Stage:Alpha]
      # [FeatureFlag:PlayerAllocationFilter]
      players:
        minAvailable: 0
        maxAvailable: 99
    - matchLabels:
        agones.dev/fleet: blue-fleet
  # defines how GameServers are organised across the cluster.
  # Options include:
  # "Packed" (default) is aimed at dynamic Kubernetes clusters, such as cloud providers, wherein we want to bin pack
  # resources
  # "Distributed" is aimed at static Kubernetes clusters, wherein we want to distribute resources across the entire
  # cluster
  scheduling: Packed
  # Optional custom metadata that is added to the game server at allocation
  # You can use this to tell the server necessary session data
  metadata:
    labels:
      mode: deathmatch
    annotations:
      map:  garden22

The spec field is the actual GameServerAllocation specification, and it is composed as follows:

Deprecated, use selectors instead. If selectors is set, this field will be ignored. required is a GameServerSelector (matchLabels. matchExpressions, gameServerState and player filters) from which to choose GameServers from.
Deprecated, use selectors instead. If selectors is set, this field will be ignored. preferred is an ordered list of preferred GameServerSelector that are optional to be fulfilled, but will be searched before the required selector. If the first selector is not matched, the selection attempts the second selector, and so on. If any of the preferred selectors are matched, the required selector is not considered. This is useful for things like smoke testing of new game servers.
selectors is an ordered list of GameServerSelector. If the first selector is not matched, the selection attempts the second selector, and so on. This is useful for things like smoke testing of new game servers.
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed. Optional.
matchExpressions is a list of label selector requirements. The requirements are ANDed. Optional.
gameServerState GameServerState specifies which State is the filter to be used when attempting to retrieve a GameServer via Allocation. Defaults to “Ready”. The only other option is “Allocated”, which can be used in conjunction with label/annotation/player selectors to retrieve an already Allocated GameServer.
counters (Alpha, “CountsAndLists” feature flag) enables filtering based on game server Counter status, such as the minimum and maximum number of active rooms. This helps in selecting game servers based on their current activity or capacity. Optional.
lists (Alpha, “CountsAndLists” feature flag) enables filtering based on game server List status, such as allowing for inclusion or exclusion of specific players. Optional.
scheduling defines how GameServers are organised across the cluster, in this case specifically when allocating GameServers for usage. “Packed” (default) is aimed at dynamic Kubernetes clusters, such as cloud providers, wherein we want to bin pack resources. “Distributed” is aimed at static Kubernetes clusters, wherein we want to distribute resources across the entire cluster. See Scheduling and Autoscaling for more details.
metadata is an optional list of custom labels and/or annotations that will be used to patch the game server’s metadata in the moment of allocation. This can be used to tell the server necessary session data
priorities (Alpha, requires CountsAndLists feature flag) alters the priority by which game GameServers are allocated by available capacity.
counters (Alpha, “CountsAndLists” feature flag) Counter actions to perform during allocation.
lists (Alpha, “CountsAndLists” feature flag) List actions to perform during allocation.

Once created the GameServerAllocation will have a status field consisting of the following:

State is the current state of a GameServerAllocation, e.g. Allocated, or UnAllocated
GameServerName is the name of the game server attached to this allocation, once the state is Allocated
Ports is a list of the ports that the game server makes available. See the GameServer Reference for more details.
Address is the primary network address where the game server can be reached.
Addresses is an array of all network addresses where the game server can be reached. It is a copy of the Node.Status.addresses field for the node the GameServer is scheduled on.
NodeName is the name of the node that the gameserver is running on.
Source is “local” unless this allocation is from a remote cluster, in which case Source is the endpoint of the remote agones-allocator. See Multi-cluster Allocation for more details.
Metadata conststs of:
- Labels containing the labels of the game server at allocation time.
- Annotations containing the annotations of the underlying game server at allocation time.
Counters (Alpha, “CountsAndLists” feature flag) is a map of CounterStatus of the game server at allocation time.
Lists (Alpha, “CountsAndLists” feature flag) is a map of ListStatus of the game server at allocation time.

Info

<p>For performance reasons, the query cache for a <code>GameServerAllocation</code> is <em>eventually consistent</em>.</p>

Usually, the cache is populated practically immediately on GameServer change, but under high load of the Kubernetes control plane, it may take some time for updates to GameServer selectable features to be populated into the cache (although this doesn’t affect the atomicity of the Allocation operation).

While Agones will do a small series of retries when an allocatable GameServer is not available in its cache, depending on your game requirements, it may be worth implementing your own more extend retry mechanism for Allocation requests for high load scenarios.

Each GameServerAllocation will allocate from a single namespace. The namespace can be specified outside of the spec, either with the --namespace flag when using the command line / kubectl or in the url when using an API call. If not specified when using the command line, the namespace will be automatically set to default.

Next Steps:

Check out the Allocator Service as a richer alternative to GameServerAllocation.

Docs: Go Game Server Client SDK

Fri, 17 May 2019 10:17:50 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GameServer	✔️
Configuration	Watch	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	✔️
Counters	SetCounterCount	✔️
Counters	IncrementCounter	✔️
Counters	DecrementCounter	✔️
Counters	SetCounterCapacity	✔️
Counters	GetCounterCapacity	✔️
Lists	AppendListValue	✔️
Lists	DeleteListValue	✔️
Lists	SetListCapacity	✔️
Lists	GetListCapacity	✔️
Lists	ListContains	✔️
Lists	GetListLength	✔️
Lists	GetListValues	✔️
Player Tracking	GetConnectedPlayers	✔️
Player Tracking	GetPlayerCapacity	✔️
Player Tracking	GetPlayerCount	✔️
Player Tracking	IsPlayerConnected	✔️
Player Tracking	PlayerConnect	✔️
Player Tracking	PlayerDisconnect	✔️
Player Tracking	SetPlayerCapacity	✔️

Installation

go get the source, directly from GitHub

Usage

Review the GoDoc for usage instructions

Docs: Limiting CPU & Memory

Thu, 03 Jan 2019 05:45:15 +0000

As a short description:

CPU Requests are limits that are applied when there is CPU congestion, and as such can burst above their set limits.
CPU Limits are hard limits on how much CPU time the particular container gets access to.

This is useful for game servers, not just as a mechanism to distribute compute resources evenly, but also as a way to advice the Kubernetes scheduler how many game server processes it is able to fit into a given node in the cluster.

It’s worth reading the Managing Compute Resources for Containers Kubernetes documentation for more details on “requests” and “limits” to both CPU and Memory, and how to configure them.

GameServers

Since the GameServer specification provides a full PodSpecTemplate, we can take advantage of both resource limits and requests in our GameServer configurations.

For example, to set a CPU limit on our GameServer configuration of 250m/0.25 of a CPU, we could do so as followed:

apiVersion: "agones.dev/v1"
kind: GameServer
metadata:
  name: "simple-game-server"
spec:
  ports:
  - name: default
    containerPort: 7654
  template:
    spec:
      containers:
      - name: simple-game-server
        image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32
        resources:
          limits:
            cpu: "250m" #this is our limit here

If you do not set a limit or request, the default is set by Kubernetes at a 100m CPU request.

SDK GameServer sidecar

You may also want to tweak the CPU request or limits on the SDK GameServer sidecar process that spins up alongside each game server container.

You can do this through the Helm configuration when installing Agones.

By default, this is set to having a CPU request value of 30m, with no hard CPU limit. This ensures that the sidecar always has enough CPU to function, but it is configurable in case a lower, or higher value is required on your clusters, or if you desire hard limit.

Docs: Fleet Autoscaler Specification

Thu, 03 Jan 2019 03:58:55 +0000

A full FleetAutoscaler specification is available below and in the example folder for reference, but here are several examples that show different autoscaling policies.

Ready Buffer Autoscaling

Fleet autoscaling with a buffer can be used to maintain a configured number of game server instances ready to serve players based on number of allocated instances in a Fleet. The buffer size can be specified as an absolute number or a percentage of the desired number of Ready game server instances over the Allocated count.

apiVersion: "autoscaling.agones.dev/v1"
kind: FleetAutoscaler
# FleetAutoscaler Metadata
# https://v1-28.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#objectmeta-v1-meta
metadata:
  name: fleet-autoscaler-example
spec:
  # The name of the fleet to attach to and control. Must be an existing Fleet in the same namespace
  # as this FleetAutoscaler
  fleetName: fleet-example
  # The autoscaling policy
  policy:
    # type of the policy. for now, only Buffer is available
    type: Buffer
    # parameters of the buffer policy
    buffer:
      # Size of a buffer of "ready" game server instances
      # The FleetAutoscaler will scale the fleet up and down trying to maintain this buffer, 
      # as instances are being allocated or terminated
      # it can be specified either in absolute (i.e. 5) or percentage format (i.e. 5%)
      bufferSize: 5
      # minimum fleet size to be set by this FleetAutoscaler. 
      # if not specified, the actual minimum fleet size will be bufferSize
      minReplicas: 10
      # maximum fleet size that can be set by this FleetAutoscaler
      # required
      maxReplicas: 20
  # The autoscaling sync strategy
  sync:
    # type of the sync. for now, only FixedInterval is available
    type: FixedInterval
    # parameters of the fixedInterval sync
    fixedInterval:
      # the time in seconds between each auto scaling
      seconds: 30

Counter and List Autoscaling

A Counter based autoscaler can be used to autoscale GameServers based on a Count and Capacity set on each of the GameServers in a Fleet to ensure there is always a buffer of available capacity available.

For example, if you have a game server that can support 10 rooms, and you want to ensure that there are always at least 5 rooms available, you could use a counter-based autoscaler with a buffer size of 5. The autoscaler would then scale the Fleet up or down based on the difference between the count of rooms across the Fleet and the capacity of rooms across the Fleet to ensure the buffer is maintained.

Counter-based FleetAutoscaler specification below and in the example folder :

apiVersion: autoscaling.agones.dev/v1
kind: FleetAutoscaler
metadata:
  name: fleet-autoscaler-counter
spec:
  fleetName: fleet-example
  policy:
    type: Counter  # Counter based autoscaling
    counter:
      # Key is the name of the Counter. Required field.
      key: rooms
      # BufferSize is the size of a buffer of counted items that are available in the Fleet (available capacity).
      # Value can be an absolute number (ex: 5) or a percentage of the Counter available capacity (ex: 5%).
      # An absolute number is calculated from percentage by rounding up. Must be bigger than 0. Required field.
      bufferSize: 5
      # MinCapacity is the minimum aggregate Counter total capacity across the fleet.
      # If BufferSize is specified as a percentage, MinCapacity is required and cannot be 0.
      # If non zero, MinCapacity must be smaller than MaxCapacity and must be greater than or equal to BufferSize.
      minCapacity: 10
      # MaxCapacity is the maximum aggregate Counter total capacity across the fleet.
      # MaxCapacity must be greater than or equal to both MinCapacity and BufferSize. Required field.
      maxCapacity: 100

A List based autoscaler can be used to autoscale GameServers based on the List length and Capacity set on each of the GameServers in a Fleet to ensure there is always a buffer of available capacity available.

For example, if you have a game server that can support 10 players, and you want to ensure that there are always room for at least 5 players across GameServers in a Fleet, you could use a list-based autoscaler with a buffer size of 5. The autoscaler would then scale the Fleet up or down based on the difference between the total length of the players and the total players capacity across the Fleet to ensure the buffer is maintained.

List-based FleetAutoscaler specification below and in the example folder :

apiVersion: autoscaling.agones.dev/v1
kind: FleetAutoscaler
metadata:
  name: fleet-autoscaler-list
spec:
  fleetName: fleet-example
  policy:
    type: List  # List based autoscaling.
    list:
      # Key is the name of the List. Required field.
      key: players
      # BufferSize is the size of a buffer based on the List capacity that is available over the current
      # aggregate List length in the Fleet (available capacity).
      # It can be specified either as an absolute value (i.e. 5) or percentage format (i.e. 5%).
      # Must be bigger than 0. Required field.
      bufferSize: 5
      # MinCapacity is the minimum aggregate List total capacity across the fleet.
      # If BufferSize is specified as a percentage, MinCapacity is required must be greater than 0.
      # If non-zero, MinCapacity must be smaller than MaxCapacity and must be greater than or equal to BufferSize.
      minCapacity: 10
      # MaxCapacity is the maximum aggregate List total capacity across the fleet.
      # MaxCapacity must be greater than or equal to both MinCapacity and BufferSize. Required field.
      maxCapacity: 100

Webhook Autoscaling

A webhook-based FleetAutoscaler can be used to delegate the scaling logic to a separate http based service. This can be useful if you want to use a custom scaling algorithm or if you want to integrate with other systems. For example, you could use a webhook-based FleetAutoscaler to scale your fleet based on data from a match-maker or player authentication system or a combination of systems.

Webhook based autoscalers have the added benefit of being able to scale a Fleet to 0 replicas, since they are able to scale up on demand based on an external signal before a GameServerAllocation is executed from a match-maker or similar system.

In order to define the path to your Webhook you can use either URL or service. Note that caBundle parameter is required if you use HTTPS for webhook FleetAutoscaler, caBundle should be omitted if you want to use HTTP webhook server.

For Webhook FleetAutoscaler below and in example folder :

apiVersion: "autoscaling.agones.dev/v1"
kind: FleetAutoscaler
metadata:
  name: webhook-fleet-autoscaler
spec:
  fleetName: simple-game-server
  policy:
    # type of the policy - this example is Webhook
    type: Webhook
    # parameters for the webhook policy - this is a WebhookClientConfig, as per other K8s webhooks
    webhook:
      # use a service, or URL
      service:
        name: autoscaler-webhook-service
        namespace: default
        path: scale
      # optional for URL defined webhooks
      # url: ""
      # caBundle:  optional, used for HTTPS webhook type
  # The autoscaling sync strategy
  sync:
    # type of the sync. for now, only FixedInterval is available
    type: FixedInterval
    # parameters of the fixedInterval sync
    fixedInterval:
      # the time in seconds between each auto scaling
      seconds: 30

See the Webhook Endpoint Specification for the specification of the incoming and outgoing JSON packet structure for the webhook endpoint.

Spec Field Reference

The spec field of the FleetAutoscaler is composed as follows:

fleetName is name of the fleet to attach to and control. Must be an existing Fleet in the same namespace as this FleetAutoscaler.
policy is the autoscaling policy
- type is type of the policy. “Buffer” and “Webhook” are available
- buffer parameters of the buffer policy type
  - bufferSize is the size of a buffer of “ready” and “reserved” game server instances. The FleetAutoscaler will scale the fleet up and down trying to maintain this buffer, as instances are being allocated or terminated. Note that “reserved” game servers could not be scaled down. It can be specified either in absolute (i.e. 5) or percentage format (i.e. 5%)
  - minReplicas is the minimum fleet size to be set by this FleetAutoscaler. if not specified, the minimum fleet size will be bufferSize if absolute value is used. When bufferSize in percentage format is used, minReplicas should be more than 0.
  - maxReplicas is the maximum fleet size that can be set by this FleetAutoscaler. Required.
- webhook parameters of the webhook policy type
  - service is a reference to the service for this webhook. Either service or url must be specified. If the webhook is running within the cluster, then you should use service. Port 8000 will be used if it is open, otherwise it is an error.
    - name is the service name bound to Deployment of autoscaler webhook. Required (see example) The FleetAutoscaler will scale the fleet up and down based on the response from this webhook server
    - namespace is the kubernetes namespace where webhook is deployed. Optional If not specified, the “default” would be used
    - path is an optional URL path which will be sent in any request to this service. (i. e. /scale)
    - port is optional, it is the port for the service which is hosting the webhook. The default is 8000 for backward compatibility. If given, it should be a valid port number (1-65535, inclusive).
  - url gives the location of the webhook, in standard URL form ([scheme://]host:port/path). Exactly one of url or service must be specified. The host should not refer to a service running in the cluster; use the service field instead. (optional, instead of service)
  - caBundle is a PEM encoded certificate authority bundle which is used to issue and then validate the webhook’s server certificate. Base64 encoded PEM string. Required only for HTTPS. If not present HTTP client would be used.
- Note: only one buffer or webhook could be defined for FleetAutoscaler which is based on the type field.
- counter parameters of the counter policy type
  - counter contains the settings for counter-based autoscaling:
    - key is the name of the counter to use for scaling decisions.
    - bufferSize is the size of a buffer of counted items that are available in the Fleet (available capacity). Value can be an absolute number or a percentage of desired game server instances. An absolute number is calculated from percentage by rounding up. Must be bigger than 0.
    - minCapacity is the minimum aggregate Counter total capacity across the fleet. If zero, MinCapacity is ignored. If non zero, MinCapacity must be smaller than MaxCapacity and bigger than BufferSize.
    - maxCapacity is the maximum aggregate Counter total capacity across the fleet. It must be bigger than both MinCapacity and BufferSize.
- list parameters of the list policy type
  - list contains the settings for list-based autoscaling:
    - key is the name of the list to use for scaling decisions.
    - bufferSize is the size of a buffer based on the List capacity that is available over the current aggregate List length in the Fleet (available capacity). It can be specified either as an absolute value or percentage format.
    - minCapacity is the minimum aggregate List total capacity across the fleet. If zero, it is ignored. If non zero, it must be smaller than MaxCapacity and bigger than BufferSize.
    - maxCapacity is the maximum aggregate List total capacity across the fleet. It must be bigger than both MinCapacity and BufferSize. Required field.
sync is autoscaling sync strategy. It defines when to run the autoscaling
- type is type of the sync. For now only “FixedInterval” is available
- fixedInterval parameters of the fixedInterval sync
  - seconds is the time in seconds between each autoscaling

Webhook Endpoint Specification

A webhook based FleetAutoscaler sends an HTTP POST request to the webhook endpoint every sync period (default is 30s) with a JSON body, and scale the target fleet based on the data that is returned.

The JSON payload that is sent is a FleetAutoscaleReview data structure and a FleetAutoscaleResponse data structure is expected to be returned.

The FleetAutoscaleResponse’s Replica field is used to set the target Fleet count with each sync interval, thereby providing the autoscaling functionality.

// FleetAutoscaleReview is passed to the webhook with a populated Request value,
// and then returned with a populated Response.
type FleetAutoscaleReview struct {
	Request  *FleetAutoscaleRequest  `json:"request"`
	Response *FleetAutoscaleResponse `json:"response"`
}

type FleetAutoscaleRequest struct {
	// UID is an identifier for the individual request/response. It allows us to distinguish instances of requests which are
	// otherwise identical (parallel requests, requests when earlier requests did not modify etc)
	// The UID is meant to track the round trip (request/response) between the Autoscaler and the WebHook, not the user request.
	// It is suitable for correlating log entries between the webhook and apiserver, for either auditing or debugging.
	UID types.UID `json:"uid""`
	// Name is the name of the Fleet being scaled
	Name string `json:"name"`
	// Namespace is the namespace associated with the request (if any).
	Namespace string `json:"namespace"`
	// The Fleet's status values
	Status v1.FleetStatus `json:"status"`
}

type FleetAutoscaleResponse struct {
	// UID is an identifier for the individual request/response.
	// This should be copied over from the corresponding FleetAutoscaleRequest.
	UID types.UID `json:"uid"`
	// Set to false if no scaling should occur to the Fleet
	Scale bool `json:"scale"`
	// The targeted replica count
	Replicas int32 `json:"replicas"`
}

// FleetStatus is the status of a Fleet
type FleetStatus struct {
	// Replicas the total number of current GameServer replicas
	Replicas int32 `json:"replicas"`
	// ReadyReplicas are the number of Ready GameServer replicas
	ReadyReplicas int32 `json:"readyReplicas"`
	// ReservedReplicas are the total number of Reserved GameServer replicas in this fleet.
	// Reserved instances won't be deleted on scale down, but won't cause an autoscaler to scale up.
	ReservedReplicas int32 `json:"reservedReplicas"`
	// AllocatedReplicas are the number of Allocated GameServer replicas
	AllocatedReplicas int32 `json:"allocatedReplicas"`
}

For Webhook Fleetautoscaler Policy either HTTP or HTTPS could be used. Switching between them occurs depending on https presence in URL or by the presence of caBundle. The example of the webhook written in Go could be found here .

It implements the scaling logic based on the percentage of allocated gameservers in a fleet.

Docs: Quickstart: Create a Fleet Autoscaler

Wed, 02 Jan 2019 06:42:33 +0000

Prerequisites

It is assumed that you have followed the instructions to Create a Game Server Fleet and you have a running fleet of game servers.

Objectives

Create a Fleet Autoscaler in Kubernetes using Agones custom resource.
Watch the Fleet scale up when allocating GameServers
Watch the Fleet scale down when shutting down allocated GameServers
Edit the autoscaler specification to apply live changes

1. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command :

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/fleetautoscaler.yaml

You should see a successful output similar to this :

fleetautoscaler.autoscaling.agones.dev/simple-game-server-autoscaler created

This has created a FleetAutoscaler record inside Kubernetes.

2. See the autoscaler status.

kubectl describe fleetautoscaler simple-game-server-autoscaler

It should look something like this:

Name:         simple-game-server-autoscaler
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"au
toscaling.agones.dev/v1","kind":"FleetAutoscaler","metadata":{"annotations":{},
"name":"simple-game-server-autoscaler","namespace":"default"},...
API Version:  autoscaling.agones.dev/v1
Kind:         FleetAutoscaler
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-10-02T15:19:58Z
  Generation:          1
  Owner References:
    API Version:           autoscaling.agones.dev/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Fleet
    Name:                  simple-game-server
    UID:                   9960762e-c656-11e8-933e-fa163e07a1d4
  Resource Version:        6123197
  Self Link:               /apis/autoscaling.agones.dev/v1/namespaces/default/fleetautoscalers/simple-game-server-autoscaler
  UID:                     9fd0efa1-c656-11e8-933e-fa163e07a1d4
Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  2
  Desired Replicas:  2
  Last Scale Time:   <nil>
  Scaling Limited:   false
Events:              <none>

You can see the status (able to scale, not limited), the last time the fleet was scaled (nil for never) and the current and desired fleet size.

The autoscaler works by changing the desired size, and the fleet creates/deletes game server instances to achieve that number. The convergence is achieved in time, which is usually measured in seconds.

3. Allocate a Game Server from the Fleet

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. In here we are only interested in triggering allocations to see the autoscaler in action.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

You should get in return the allocated game server details, which should end with something like:

status:
  address: 34.94.118.237
  gameServerName: simple-game-server-v6jwb-6bzkz
  nodeName: gke-test-cluster-default-f11755a7-5km3
  ports:
  - name: default
    port: 7832
  state: Allocated

Note the address and port, you might need them later to connect to the server.

4. See the autoscaler in action

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler simple-game-server-autoscaler

The last part should look something like this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  3
  Desired Replicas:  3
  Last Scale Time:   2018-10-02T16:00:02Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  2m    fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 3

You can see that the fleet size has increased, the autoscaler having compensated for the allocated instance. Last Scale Time has been updated, and a scaling event has been logged.

Double-check the actual number of game server instances and status by running

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE       ADDRESS        PORT     NODE        AGE
simple-game-server-mzhrl-hz8wk   Allocated   10.30.64.99    7131     minikube    5m
simple-game-server-mzhrl-k6jg5   Ready       10.30.64.100   7243     minikube    5m  
simple-game-server-mzhrl-n2sk2   Ready       10.30.64.168   7658     minikube    5m

5. Shut the allocated instance down

Since we’ve only got one allocation, we’ll just grab the details of the IP and port of the only allocated GameServer:

kubectl get gameservers | grep Allocated | awk '{print $3":"$4 }'

This should output your Game Server IP address and port. (eg 10.130.65.208:7936)

You can now communicate with the GameServer:

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

6. See the fleet scaling down

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler simple-game-server-autoscaler

It should look something like this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  3
  Desired Replicas:  2
  Last Scale Time:   2018-10-02T16:09:02Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  9m    fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 3
  Normal  AutoScalingFleet  45s   fleetautoscaler-controller  Scaling fleet simple-game-server from 3 to 2

You can see that the fleet size has decreased, the autoscaler adjusting to game server instance being de-allocated, the Last Scale Time and the events have been updated. Note that simple-game-server game server instance you just closed earlier might stay a bit in ‘Unhealthy’ state (and its pod in ‘Terminating’ until it gets removed.

Double-check the actual number of game server instances and status by running

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE     ADDRESS        PORT    NODE       AGE
simple-game-server-mzhrl-k6jg5   Ready     10.30.64.100   7243    minikube   5m
simple-game-server-mzhrl-t7944   Ready     10.30.64.168   7561    minikube   5m

7. Change autoscaling parameters

We can also change the configuration of the FleetAutoscaler of the running Fleet, and have the changes applied live, without interruptions of service.

Run kubectl edit fleetautoscaler simple-game-server-autoscaler and set the bufferSize field to 5.

Let’s look at the list of game servers again. Run watch kubectl get gs until you can see that are 5 ready server instances:

NAME                             STATE     ADDRESS        PORT    NODE         AGE
simple-game-server-mzhrl-7jpkp   Ready     10.30.64.100   7019    minikube     5m
simple-game-server-mzhrl-czt8v   Ready     10.30.64.168   7556    minikube     5m
simple-game-server-mzhrl-k6jg5   Ready     10.30.64.100   7243    minikube     5m
simple-game-server-mzhrl-nb8h2   Ready     10.30.64.168   7357    minikube     5m
simple-game-server-mzhrl-qspb6   Ready     10.30.64.99    7859    minikube     5m
simple-game-server-mzhrl-zg9rq   Ready     10.30.64.99    7745    minikube     5m

Note

If you want to update a Fleet which has RollingUpdate replacement strategy and is controlled by a FleetAutoscaler:

With kubectl apply: you should omit replicas parameter in a Fleet Spec before re-applying the Fleet configuration.
With kubectl edit: you should not change the replicas parameter in the Fleet Spec when updating other field parameters.

If you follow the rules above, then the maxSurge and maxUnavailable parameters will be used as the RollingUpdate strategy updates your Fleet. Otherwise the Fleet would be scaled according to Fleet replicas parameter first and only after a certain amount of time it would be rescaled to fit FleetAutoscaler BufferSize parameter.

You could also check the behaviour of the Fleet with Fleetautoscaler on a test Fleet to preview what would occur in your production environment.

Next Steps

Read the advanced Scheduling and Autoscaling guide, for more details on autoscaling.

If you want to use your own GameServer container make sure you have properly integrated the Agones SDK.

Docs: Azure Kubernetes Service

Mon, 01 Jan 0001 00:00:00 +0000

Choosing your shell

You can use either Azure Cloud Shell or install the Azure CLI on your local shell in order to install AKS in your own Azure subscription. Cloud Shell comes preinstalled with az and kubectl utilities whereas you need to install them locally if you want to use your local shell. If you use Windows 10, you can use the WIndows Subsystem for Windows as well.

Creating the AKS cluster

If you are using Azure CLI from your local shell, you need to log in to your Azure account by executing the az login command and following the login procedure.

Here are the steps you need to follow to create a new AKS cluster (additional instructions and clarifications are listed here):

# Declare necessary variables, modify them according to your needs
AKS_RESOURCE_GROUP=akstestrg     # Name of the resource group your AKS cluster will be created in
AKS_NAME=akstest                 # Name of your AKS cluster
AKS_LOCATION=westeurope          # Azure region in which you'll deploy your AKS cluster

# Create the Resource Group where your AKS resource will be installed
az group create --name $AKS_RESOURCE_GROUP --location $AKS_LOCATION

# Create the AKS cluster - this might take some time. Type 'az aks create -h' to see all available options

# The following command will create a four Node AKS cluster. Node size is Standard A1 v1 and Kubernetes version is 1.28. Plus, SSH keys will be generated for you, use --ssh-key-value to provide your values
az aks create --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME --node-count 4 --generate-ssh-keys --node-vm-size Standard_A4_v2 --kubernetes-version 1.28 --enable-node-public-ip

# Install kubectl
sudo az aks install-cli

# Get credentials for your new AKS cluster
az aks get-credentials --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME

Alternatively, you can use the Azure Portal to create a new AKS cluster (instructions).

Allowing UDP traffic

For Agones to work correctly, we need to allow UDP traffic to pass through to our AKS cluster. To achieve this, we must update the NSG (Network Security Group) with the proper rule. A simple way to do that is:

Log in to the Azure Portal
Find the resource group where the AKS(Azure Kubernetes Service) resources are kept, which should have a name like MC_resourceGroupName_AKSName_westeurope. Alternative, you can type az resource show --namespace Microsoft.ContainerService --resource-type managedClusters -g $AKS_RESOURCE_GROUP -n $AKS_NAME -o json | jq .properties.nodeResourceGroup
Find the Network Security Group object, which should have a name like aks-agentpool-********-nsg (ie. aks-agentpool-55978144-nsg for dns-name-prefix agones)
Select Inbound Security Rules
Select Add to create a new Rule with UDP as the protocol and 7000-8000 as the Destination Port Ranges. Pick a proper name and leave everything else at their default values

Alternatively, you can use the following command, after modifying the RESOURCE_GROUP_WITH_AKS_RESOURCES and NSG_NAME values:

az network nsg rule create \
  --resource-group RESOURCE_GROUP_WITH_AKS_RESOURCES \
  --nsg-name NSG_NAME \
  --name AgonesUDP \
  --access Allow \
  --protocol Udp \
  --direction Inbound \
  --priority 520 \
  --source-port-range "*" \
  --destination-port-range 7000-8000

Getting Public IPs to Nodes

Kubernetes version prior to 1.18.19, 1.19.11 and 1.20.7

To find a resource’s public IP, search for Virtual Machine Scale Sets -> click on the set name(inside MC_resourceGroupName_AKSName_westeurope group) -> click Instances -> click on the instance name -> view Public IP address.

To get public IP via API look here.

For more information on Public IPs for VM NICs, see this document.

Kubernetes version starting 1.18.19, 1.19.11 and 1.20.7

Virtual Machines public IP is available directly in Kubernetes EXTERNAL-IP.

Next Steps

Continue to Install Agones.

Docs: Local Game Server

Tue, 19 Feb 2019 08:00:00 +0000

You can register a local game server with Agones. This means you can run an experimental build of your game server in the Agones environment without the need of packaging and deploying it to a fleet. This allows you to quickly iterate on your game server code while still being able to plugin to your Agones environment.

Register your server with Agones

To register your local game server you’ll need to know the IP address of the machine running it and the port. With that you’ll create a game server config like the one below.

apiVersion: "agones.dev/v1"
kind: GameServer
metadata:
  name: my-local-server
  annotations:
    # Causes Agones to register your local game server at 192.1.1.2, replace with your server's IP address.
    agones.dev/dev-address: "192.1.1.2"
spec:
  ports:
  - name: default
    portPolicy: Static
    hostPort: 17654
    containerPort: 17654
  # The following is ignored but required due to validation.
  template:
    spec:
      containers:
      - name: simple-game-server
        image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32

Once you save this to a file make sure you have kubectl configured to point to your Agones cluster and then run kubectl apply -f dev-gameserver.yaml. This will register your server with Agones.

Local Game Servers has a few limitations:

PortPolicy must be Static.
The game server is not managed by Agones. Features like autoscaling, replication, etc are not available.

When you are finished working with your server, you can remove the registration with kubectl delete -f dev-gameserver.yaml

Next Steps:

Review the specification of GameServer.
Read about GameServer allocation.
- Review the flow of how allocation is done.
- Review the specification of GameServerAllocation.
- Check out the Allocator Service as a richer alternative to GameServerAllocation.
Learn how to connect your local development game server binary into a running Agones Kubernetes cluster for even more live development options with an out of cluster dev server.

Docs: Latency Testing with Multiple Clusters

Thu, 03 Jan 2019 01:20:30 +0000

To make latency testing easier, Agones installs with a simple ping service with both HTTP and UDP services that can be called for the purpose of timing how long the roundtrip takes for information to be returned from either of these services.

Installing

By default, Agones installs Kubernetes Services for both HTTP and the UDP ping endpoints. These can be disabled entirely, or disabled individually. See the Helm install guide for the parameters to pass through, as well as configuration options.

The ping services as all installed under the agones-system namespace.

HTTP Service

This exposes an endpoint that returns a simple text HTTP response on request to the root “/” path. By default this is ok, but it can be configured via the agones.ping.http.response parameter.

This could be useful for providing clusters with unique lookup names, such that clients are able to identify clusters from their responses.

To lookup the details of this service, run kubectl describe service agones-ping-http-service --namespace=agones-system

UDP Service

The UDP ping service is a rate limited UDP echo service that returns the udp packet that it receives to its designated sender.

Since UDP sender details can be spoofed, this service is rate limited to 20 requests per second, per sender address, per running instance (default is 2).

This rate limit can be raised or lowered via the Helm install parameter agones.ping.udp.rateLimit.

UDP packets are also limited to 1024 bytes in size.

To lookup the details of this service, run kubectl describe service agones-ping-udp-service --namespace=agones-system

Client side tooling

We deliberately didn’t provide any game client libraries, as all major languages and engines have capabilities to send HTTP requests as well as UDP packets.

Docs: Quickstart: Create a Fleet Autoscaler with Webhook Policy

Wed, 02 Jan 2019 06:42:44 +0000

In some cases, your game servers may need to use custom logic for scaling your fleet that is more complex than what can be expressed using the Buffer policy in the fleetautoscaler. This guide shows how you can extend Agones with an autoscaler webhook to implement a custom autoscaling policy.

When you use an autoscaler webhook the logic computing the number of target replicas is delegated to an external HTTP/S endpoint, such as one provided by a Kubernetes deployment and service in the same cluster (as shown in the examples below). The fleetautoscaler will send a request to the webhook autoscaler’s /scale endpoint every sync period (currently 30s) with a JSON body, and scale the target fleet based on the data that is returned.

Chapter 1 Configuring HTTP fleetautoscaler webhook

Prerequisites

It is assumed that you have completed the instructions to Create a Game Server Fleet and have a running fleet of game servers.

Objectives

Run a fleet
Deploy the Webhook Pod and service for autoscaling
Create a Fleet Autoscaler with Webhook policy type in Kubernetes using Agones custom resource
Watch the Fleet scales up when allocating GameServers
Watch the Fleet scales down after GameServer shutdown

1. Deploy the fleet

Run a fleet in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/fleet.yaml

2. Deploy a Webhook service for autoscaling

In this step we would deploy an example webhook that will control the size of the fleet based on allocated gameservers portion in a fleet. You can see the source code for this example webhook server here . The fleetautoscaler would trigger this endpoint every 30 seconds. More details could be found also here . We need to create a pod which will handle HTTP requests with json payload FleetAutoscaleReview and return back it with FleetAutoscaleResponse populated.

The Scale flag and Replicas values returned in the FleetAutoscaleResponse tells the FleetAutoscaler what target size the backing Fleet should be scaled up or down to. If Scale is false - no scaling occurs.

Run next command to create a service and a Webhook pod in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/autoscaler-webhook/autoscaler-service.yaml

To check that it is running and liveness probe is fine:

kubectl describe pod autoscaler-webhook

Name:           autoscaler-webhook-86944884c4-sdtqh
Namespace:      default
Node:           gke-test-cluster-default-1c5dec79-h0tq/10.138.0.2
...
Status:         Running

3. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/webhookfleetautoscaler.yaml

You should see a successful output similar to this:

fleetautoscaler.autoscaling.agones.dev "webhook-fleet-autoscaler" created

This has created a FleetAutoscaler record inside Kubernetes. It has the link to Webhook service we deployed above.

4. See the fleet and autoscaler status.

In order to track the list of gameservers which run in your fleet you can run this command in a separate terminal tab:

 watch "kubectl get gs -n default"

In order to get autoscaler status use the following command:

kubectl describe fleetautoscaler webhook-fleet-autoscaler

It should look something like this:

Name:         webhook-fleet-autoscaler
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":
"autoscaling.agones.dev/v1","kind":"FleetAutoscaler","metadata":{"annotations"
:{},"name":"webhook-fleet-autoscaler","namespace":"default...
API Version:  autoscaling.agones.dev/v1
Kind:         FleetAutoscaler
etadata:
  Cluster Name:
  Creation Timestamp:  2018-12-22T12:52:23Z
  Generation:          1
  Resource Version:    2274579
  Self Link:           /apis/autoscaling.agones.dev/v1/namespaces/default/fleet
autoscalers/webhook-fleet-autoscaler
  UID:                 6d03eae4-05e8-11e9-84c2-42010a8a01c9
Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Service:
        Name:       autoscaler-webhook-service
        Namespace:  default
        Path:       scale
      URL:
Status:
  Able To Scale:     true
  Current Replicas:  2
  Desired Replicas:  2
  Last Scale Time:   <nil>
  Scaling Limited:   false
Events:              <none>

You can see the status (able to scale, not limited), the last time the fleet was scaled (nil for never), current and desired fleet size.

The autoscaler makes a query to a webhoook service deployed on step 1 and on response changing the target Replica size, and the fleet creates/deletes game server instances to achieve that number. The convergence is achieved in time, which is usually measured in seconds.

5. Allocate Game Servers from the Fleet to trigger scale up

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. Here we only interested in triggering allocations to see the autoscaler in action.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

You should get in return the allocated game server details, which should end with something like:

status:
  address: 34.94.118.237
  gameServerName: simple-game-server-v6jwb-6bzkz
  nodeName: gke-test-cluster-default-f11755a7-5km3
  ports:
  - name: default
    port: 7832
  state: Allocated

Note the address and port, you might need them later to connect to the server.

Run the kubectl command one more time so that we have both servers allocated:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

6. Check new Autoscaler and Fleet status

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler webhook-fleet-autoscaler

The last part should look similar to this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Service:
        Name:       autoscaler-webhook-service
        Namespace:  default
        Path:       scale
      URL:
Status:
  Able To Scale:     true
  Current Replicas:  4
  Desired Replicas:  4
  Last Scale Time:   2018-12-22T12:53:47Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  35s   fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4

You can see that the fleet size has increased in particular case doubled to 4 gameservers (based on our custom logic in our webhook), the autoscaler having compensated for the two allocated instances. Last Scale Time has been updated and a scaling event has been logged.

Double-check the actual number of game server instances and status by running:

 kubectl get gs -n default

This will get you a list of all the current GameServers and their Status > State.

NAME                     STATE       ADDRESS         PORT     NODE        AGE
simple-game-server-dmkp4-8pkk2   Ready       35.247.13.175   7386     minikube     5m
simple-game-server-dmkp4-b7x87   Allocated   35.247.13.175   7219     minikube     5m
simple-game-server-dmkp4-r4qtt   Allocated   35.247.13.175   7220     minikube     5m
simple-game-server-dmkp4-rsr6n   Ready       35.247.13.175   7297     minikube     5m

7. Check downscaling using Webhook Autoscaler policy

Based on our custom webhook deployed earlier, if the fraction of allocated replicas in whole Replicas count would be less than threshold (0.3) then the fleet would scale down by scaleFactor, in our example by 2.

Note that the example webhook server has a limitation that it would not decrease fleet replica count under minReplicasCount, which is equal to 2.

We need to run EXIT command on one gameserver (Use IP address and port of the allocated gameserver from the previous step) in order to decrease the number of allocated gameservers in a fleet (<0.3).

nc -u 35.247.13.175 7220
EXIT

Server would be in shutdown state. Wait about 30 seconds. Then you should see scaling down event in the output of next command:

kubectl describe fleetautoscaler webhook-fleet-autoscaler

You should see these lines in events:

  Normal   AutoScalingFleet  11m                fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4
  Normal   AutoScalingFleet  1m                 fleetautoscaler-controller  Scaling fleet simple-game-server from 4 to 2

And get gameservers command output:

kubectl get gs -n default

NAME                             STATUS      ADDRESS          PORT     NODE       AGE
simple-game-server-884fg-6q5sk   Ready       35.247.117.202   7373     minikube   5m
simple-game-server-884fg-b7l58   Allocated   35.247.117.202   7766     minikube   5m

8. Cleanup

You can delete the autoscaler service and associated resources with the following commands.

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/autoscaler-webhook/autoscaler-service.yaml

Removing the fleet:

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/fleet.yaml

Chapter 2 Configuring HTTPS fleetautoscaler webhook with CA Bundle

Objectives

Using TLS and a certificate authority (CA) bundle we can establish trusted communication between Fleetautoscaler and an HTTPS server running the autoscaling webhook that controls the size of the fleet (Replicas count). The certificate of the autoscaling webhook must be signed by the CA provided in fleetautoscaler yaml configuration file. Using TLS eliminates the possibility of a man-in-the-middle attack between the fleetautoscaler and the autoscaling webhook.

1. Deploy the fleet

Run a fleet in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/fleet.yaml

2. Create X509 Root and Webhook certificates

The procedure of generating a Self-signed CA certificate is as follows:

The first step is to create the private root key:

openssl genrsa -out rootCA.key 2048

The next step is to self-sign this certificate:

openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem

This will start an interactive script that will ask you for various bits of information. Fill it out as you see fit.

Every webhook that you wish to install a trusted certificate will need to go through this process. First, just like with the root CA step, you’ll need to create a private key (different from the root CA):

openssl genrsa -out webhook.key 2048

Next create configuration file cert.conf for the certificate signing request:

[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[req_distinguished_name]
CN = autoscaler-tls-service.default.svc
[v3_req]
keyUsage = digitalSignature
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = autoscaler-tls-service.default.svc

Generate the certificate signing request, use valid hostname which in this case will be autoscaler-tls-service.default.svc as Common Name (eg, fully qualified host name) as well as DNS.1 in the alt_names section of the config file.

Check the Kubernetes documentation to see how Services get assigned DNS entries.

openssl req -new -out webhook.csr -key webhook.key -config cert.conf

Once that’s done, you’ll sign the CSR, which requires the CA root key:

openssl x509 -req -in webhook.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out webhook.crt -days 500 -sha256 -extfile cert.conf -extensions v3_req

This would generate webhook.crt certificate

Add secret which later would be mounted to autoscaler-webhook-tls pod.

kubectl create secret tls autoscalersecret --cert=webhook.crt --key=webhook.key

You need to put Base64-encoded string into caBundle field in your fleetautoscaler yaml configuration:

wget https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/webhookfleetautoscalertls.yaml
sed --in-place -e "s/\$CA_BUNDLE/$( cat ./rootCA.pem |  base64 -w0)/" webhookfleetautoscalertls.yaml

3. Deploy a Webhook service for autoscaling

Run next command to create a service and a Webhook pod in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/autoscaler-webhook/autoscaler-service-tls.yaml

To check that it is running and liveness probe is fine:

kubectl describe pod autoscaler-webhook-tls

Wait for the Running status results:

Name:               autoscaler-webhook-tls-f74c9bff7-ssrsc
Namespace:          default
...
Status:         Running

4. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command (caBundle should be set properly on Step 2):

kubectl apply -f ./webhookfleetautoscalertls.yaml

5. See the fleet and autoscaler status.

In order to track the list of gameservers which run in your fleet you can run this command in a separate terminal tab:

 watch "kubectl get gs -n default"

6. Allocate two Game Servers from the Fleet to trigger scale up

for i in {0..1} ; do kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/gameserverallocation.yaml -o yaml ; done

7. Check new Autoscaler and Fleet status

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler  webhook-fleetautoscaler-tls

The last part should look similar to this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Ca Bundle:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1RENDQWFBQ0NRQ29kcEFNbTlTd0pqQU5CZ2txaGtpRzl3MEJBUXNGQURBZU1Rc3dDUVlEVlFRR0V3SlYKVXpFUE1BMEdBMVVFQ3d3R1FXZHZibVZ6TUI0WERURTVNREV3TkRFeE5URTBORm9YRFRJeE1UQXlOREV4TlRFMApORm93SGpFTE1Ba0dBMVVFQmhNQ1ZWTXhEekFOQmdOVkJBc01Ca0ZuYjI1bGN6Q0NBU0l3RFFZSktvWklodmNOCkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFOQ0h5dndDOTZwZDlTdkFhMUIvRWg2ekcxeDBLS1dPaVhtNzhJcngKKzZ5WHd5YVpsMVo1cVExbUZoOThMSGVZUmQwWVgzRTJnelZ5bFpvUlUra1ZESzRUc0VzV0tNUFVpdVo0MUVrdApwbythbEN6alAyaXZzRGZaOGEvdnByL3dZZ2FrWGtWalBUaGpKUk9xTnFIdWROMjZVcUFJYnNOTVpoUkxkOVFFCnFLSjRPNmFHNVMxTVNqZFRGVHFlbHJiZitDcXNKaHltZEIzZmxGRUVvdXExSmoxS0RoQjRXWlNTbS9VSnpCNkcKNHUzY3BlQm1jTFVRR202ZlFHb2JFQSt5SlpMaEVXcXBrd3ZVZ2dCNmRzWE8xZFNIZXhhZmlDOUVUWGxVdFRhZwo1U2JOeTVoYWRWUVV3Z253U0J2djR2R0t1UUxXcWdXc0JyazB5Wll4Sk5Bb0V5RUNBd0VBQVRBTkJna3Foa2lHCjl3MEJBUXNGQUFPQ0FRRUFRMkgzaWJRcWYzQTNES2l1eGJISURkbll6TlZ2Z0dhRFpwaVZyM25ocm55dmxlNVgKR09hRm0rMjdRRjRWV29FMzZDTGhYZHpEWlM4bEpIY09YUW5KOU83Y2pPYzkxVmh1S2NmSHgwS09hU1oweVNrVAp2bEtXazlBNFdoNGE0QXFZSlc3Z3BUVHR1UFpydnc4VGsvbjFaWEZOYVdBeDd5RU5OdVdiODhoNGRBRDVaTzRzCkc5SHJIdlpuTTNXQzFBUXA0Q3laRjVyQ1I2dkVFOWRkUmlKb3IzM3pLZTRoRkJvN0JFTklZZXNzZVlxRStkcDMKK0g4TW5LODRXeDFUZ1N5Vkp5OHlMbXFpdTJ1aThjaDFIZnh0OFpjcHg3dXA2SEZLRlRsTjlBeXZUaXYxYTBYLwpEVTk1eTEwdi9oTlc0WHpuMDJHNGhrcjhzaUduSEcrUEprT3hBdz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
      Service:    <nil>
      URL:        https://autoscaler-tls-service.default.svc:8000/scale
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  5s   fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4

Double-check the actual number of game server instances and status by running:

 kubectl get gs -n default

This will get you a list of all the current GameServers and their Status > State.

NAME                     STATE       ADDRESS         PORT      NODE      AGE
simple-game-server-njmr7-2t4nx   Ready       35.203.159.68   7330      minikube   1m
simple-game-server-njmr7-65rp6   Allocated   35.203.159.68   7294      minikube   4m

8. Cleanup

You can delete the autoscaler service and associated resources with the following commands.

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/autoscaler-webhook/autoscaler-service-tls.yaml

Removing x509 key secret:

kubectl delete secret autoscalersecret

Removing the fleet:

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/simple-game-server/fleet.yaml

Comments

Note that secure communication has been established and we can trust that communication between the fleetautoscaler and the autoscaling webhook. If you need to run the autoscaling webhook outside of the Kubernetes cluster, you can use another root certificate authority as long as you put it into the caBundle parameter in fleetautoscaler configuration (in pem format, base64-encoded).

Troubleshooting Guide

If you run into problems with the configuration of your fleetautoscaler and webhook service the easiest way to debug them is to run:

kubectl describe fleetautoscaler <FleetAutoScalerName>

and inspect the events at the bottom of the output.

Common error messages.

If you have configured the wrong service Path for the FleetAutoscaler you will see a message like

Error calculating desired fleet size on FleetAutoscaler simple-fleet-r7fdv-autoscaler. Error: bad status code 404 from the server: https://autoscaler-tls-service.default.svc:8000/scale

If you are using a hostname other than autoscaler-tls-service.default.svc as the Common Name (eg, fully qualified host name) when creating a certificate using openssl tool you will see a message like

Post https://autoscaler-tls-service.default.svc:8000/scale: x509: certificate is not valid for any names, but wanted to match autoscaler-tls-service.default.svc

If you see errors like the following in autoscaler-webhook-tls pod logs:

http: TLS handshake error from 10.48.3.125:33374: remote error: tls: bad certificate

Then there could be an issue with your ./rootCA.pem.

You can repeat the process from step 2, in order to fix your certificates setup.

Next Steps

Read the advanced Scheduling and Autoscaling guide, for more details on autoscaling.

If you want to use your own GameServer container make sure you have properly integrated the Agones SDK.

Docs: Reusing Allocated GameServers for more than one game session

Wed, 01 Sep 2021 00:00:00 +0000

Having a GameServer terminate after a single player session is better for packing and optimisation of infrastructure usage, as well as safety to ensure the process returns to an absolute zero state.

However, depending on the GameServer startup time, or other factors there may be reasons you wish to reuse a GameServer for n number of sessions before finally shutting it down.

The “magic trick” to this is knowing that the GameServer process can call SDK.Ready() to return to a Ready state after the GameServer has been allocated.

It is then up to the game developer to ensure that the game server process returns to a zero state once a game session has been completed.

Next Steps

Have a look at all commands the Client SDK provides.
If you aren’t familiar with the term Pod, this shouldw provide a reference.

Docs: C# Game Server Client SDK

Thu, 28 May 2020 00:00:00 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GetGameServer	✔️
Configuration	WatchGameServer	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	✔️
Counters	SetCounterCount	✔️
Counters	IncrementCounter	✔️
Counters	DecrementCounter	✔️
Counters	SetCounterCapacity	✔️
Counters	GetCounterCapacity	✔️
Lists	AppendListValue	✔️
Lists	DeleteListValue	✔️
Lists	SetListCapacity	✔️
Lists	GetListCapacity	✔️
Lists	ListContains	✔️
Lists	GetListLength	✔️
Lists	GetListValues	✔️
Player Tracking	GetConnectedPlayers	✔️
Player Tracking	GetPlayerCapacity	✔️
Player Tracking	GetPlayerCount	✔️
Player Tracking	IsPlayerConnected	✔️
Player Tracking	PlayerConnect	✔️
Player Tracking	PlayerDisconnect	✔️
Player Tracking	SetPlayerCapacity	✔️

Download

Download the source directly from GitHub .

Install using NuGet

Download the nuget package directly
Install the latest version using the Package Manager: Install-Package AgonesSDK
Install the latest version using the .NET CLI: dotnet add package AgonesSDK

To select a specific version, append --version, for example: --version 1.8.0 to either commands.

Prerequisites

.Net Standard 2.0 compliant framework.

Usage

Reference the SDK in your project & create a new instance of the SDK wrapper:

Initialization

To use the AgonesSDK, you will need to import the namespace by adding using Agones; at the beginning of your relevant files.

var agones = new AgonesSDK();

Connection

To connect to the SDK server, either locally or when running on Agones, run the ConnectAsync() method. This will wait for up to 30 seconds if the SDK server has not yet started and the connection cannot be made, and will return false if there was an issue connecting.

bool ok = await agones.ConnectAsync();

Ready

To mark the game server as ready to receive player connections, call the async method ReadyAsync().

async void SomeMethod()
{
    var status = await agones.ReadyAsync();
}

Health

To send Health pings, call the async method HealthAsync()

await agones.HealthAsync();

GetGameServer

To get the details on the backing GameServer call GetGameServerAsync().

Will return null if there is an error in retrieving the GameServer record.

var gameserver = await agones.GetGameServerAsync();

Reserve

To mark the GameServer as Reserved for a duration call ReserveAsync(long duration).

long duration = 30;
var status = await agones.ReserveAsync(duration);

ShutDown

To mark that the game session is completed and the game server should be shut down call ShutdownAsync().

var status = await agones.ShutdownAsync();

SetAnnotation & SetLabel

Similarly SetAnnotation(string key, string value) and SetLabel(string key, string value) are async methods that perform an action & return a Status object.

WatchGameServer

To watch when the backing GameServer configuration changes call WatchGameServer(callback), where the delegate function callback of type Action<GameServer> will be executed every time the GameServer configuration changes. This process is non-blocking internally.

agonesSDK.WatchGameServer((gameServer) => { Console.WriteLine($"Server - Watch {gameServer}");});

Counters And Lists

Warning

The Counters And Lists feature is currently Alpha, not enabled by default, and may change in the future.

Use the FeatureGate CountsAndLists to enable and test this feature.

See the Feature Gate documentation for details on how to enable features.

Counters

Alpha: GetCounterCount

Returns the Count for a Counter, given the Counter’s key (name). Will error if the key was not predefined in the GameServer resource on creation.

string key = "rooms";
long count = await agones.Alpha().GetCounterCountAsync(key);

Alpha: SetCounterCount

Sets a count to the given value. Use with care, as this will overwrite any previous invocations’ value. Cannot be greater than Capacity.

string key = "rooms";
long amount = 0;
await agones.Alpha().SetCounterCountAsync(key, amount);

Alpha: IncrementCounter

Increases a counter by the given nonnegative integer amount. Will execute the increment operation against the current CRD value. Will max at max(int64). Will error if the key was not predefined in the GameServer resource on creation. Errors if the count is at the current capacity (to the latest knowledge of the SDK), and no increment will occur.

Note: A potential race condition here is that if count values are set from both the SDK and through the K8s API (Allocation or otherwise), since the SDK append operation back to the CRD value is batched asynchronous any value incremented past the capacity will be silently truncated.

string key = "rooms";
long amount = 1;
await agones.Alpha().IncrementCounterAsync(key, amount);

Alpha: DecrementCounter

Decreases the current count by the given nonnegative integer amount. The Counter Will not go below 0. Will execute the decrement operation against the current CRD value. Errors if the count is at 0 (to the latest knowledge of the SDK), and no decrement will occur.

string key = "rooms";
long amount = 2;
await agones.Alpha().DecrementCounterAsync(key, amount);

Alpha: SetCounterCapacity

Sets the capacity for the given Counter. A capacity of 0 is no capacity.

string key = "rooms";
long amount = 0;
await agones.Alpha().SetCounterCapacityAsync(key, amount);

Alpha: GetCounterCapacity

Returns the Capacity for a Counter, given the Counter’s key (name). Will error if the key was not predefined in the GameServer resource on creation.

string key = "rooms";
long count = await agones.Alpha().GetCounterCapacityAsync(key);

Lists

Alpha: AppendListValue

Appends a string to a List’s values list, given the List’s key (name) and the string value. Will error if the string already exists in the list. Will error if the key was not predefined in the GameServer resource on creation. Will error if the list is already at capacity.

string key = "players";
string value = "player1";
await agones.Alpha().AppendListValueAsync(key, value);

Alpha: DeleteListValue

DeleteListValue removes a string from a List’s values list, given the List’s key (name) and the string value. Will error if the string does not exist in the list. Will error if the key was not predefined in the GameServer resource on creation.

string key = "players";
string value = "player2";
await agones.Alpha().DeleteListValueAsync(key, value);

Alpha: SetListCapacity

Sets the capacity for a given list. Capacity must be between 0 and 1000. Will error if the key was not predefined in the GameServer resource on creation.

string key = "players";
long amount = 1000;
await agones.Alpha().SetListCapacityAsync(key, amount);

Alpha: GetListCapacity

Returns the Capacity for a List, given the List’s key (name). Will error if the key was not predefined in the GameServer resource on creation.

string key = "players";
long amount = await agones.Alpha().GetListCapacityAsync(key);

Alpha: ListContains

Returns if a string exists in a List’s values list, given the List’s key (name) and the string value. Search is case-sensitive. Will error if the key was not predefined in the GameServer resource on creation.

string key = "players";
string value = "player3";
bool contains = await agones.Alpha().ListContainsAsync(key, value);

Alpha: GetListLength

GetListLength returns the length of the Values list for a List, given the List’s key (name). Will error if the key was not predefined in the GameServer resource on creation.

string key = "players";
int listLength = await agones.Alpha().GetListLengthAsync(key);

Alpha: GetListValues

Returns the <IList> Values for a List, given the List’s key (name). Will error if the key was not predefined in the GameServer resource on creation.

string key = "players";
List<string> values = await agones.Alpha().GetListValuesAsync(key);

Player Tracking

Warning

The Player Tracking feature is currently Alpha, not enabled by default, and may change in the future.

Use the FeatureGate PlayerTracking to enable and test this feature.

See the Feature Gate documentation for details on how to enable features.

Alpha: PlayerConnect

bool ok = await agones.Alpha().PlayerConnectAsync(playerId);

Alpha: PlayerDisconnect

bool ok = await agones.Alpha().PlayerDisconnectAsync(playerId);

Alpha: SetPlayerCapacity

Update the GameServer.Status.Players.Capacity value with a new capacity.

var capacity = 100;
var status = await agones.Alpha().SetPlayerCapacityAsync(capacity);

Alpha: GetPlayerCapacity

long cap = await agones.Alpha().GetPlayerCapacityAsync();

Alpha: GetPlayerCount

Returns the current player count

long count = await agones.Alpha().GetPlayerCountAsync();

Alpha: IsPlayerConnected

This returns if the playerID is currently connected to the GameServer. This is always accurate, even if the value hasn’t been updated to the GameServer status yet.

var playerId = "player1";
bool isConnected = await agones.Alpha().IsPlayerConnectedAsync(playerId);

Remarks

All requests other than ConnectAsync will wait for up to 15 seconds before giving up, time to wait can also be set in the constructor.
Default host & port are localhost:9357
Methods that do not return a data object such as GameServer will return a gRPC Grpc.Core.Status object. To check the state of the request, check Status.StatusCode & Status.Detail. Ex:

if(status.StatusCode == StatusCode.OK)
    //do stuff

Docs: Node.js Game Server Client SDK

Sun, 24 Feb 2019 15:56:57 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GetGameServer	✔️
Configuration	WatchGameServer	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	❌
Counters	SetCounterCount	❌
Counters	IncrementCounter	❌
Counters	DecrementCounter	❌
Counters	SetCounterCapacity	❌
Counters	GetCounterCapacity	❌
Lists	AppendListValue	❌
Lists	DeleteListValue	❌
Lists	SetListCapacity	❌
Lists	GetListCapacity	❌
Lists	ListContains	❌
Lists	GetListLength	❌
Lists	GetListValues	❌
Player Tracking	GetConnectedPlayers	✔️
Player Tracking	GetPlayerCapacity	✔️
Player Tracking	GetPlayerCount	✔️
Player Tracking	IsPlayerConnected	✔️
Player Tracking	PlayerConnect	✔️
Player Tracking	PlayerDisconnect	✔️
Player Tracking	SetPlayerCapacity	✔️

Prerequisites

Node.js >= 10.13.0

Usage

Add the agones dependency to your project:

npm install @google-cloud/agones-sdk

If you need to download the source, rather than install from NPM, you can find it on GitHub .

To begin working with the SDK, create an instance of it.

const AgonesSDK = require('@google-cloud/agones-sdk');

let agonesSDK = new AgonesSDK();

To connect to the SDK server, either local or when running on Agones, run the async method sdk.connect(), which will resolve once connected or reject on error or if no connection can be made after 30 seconds.

await agonesSDK.connect();

To send a health check ping call health(errorCallback). The error callback is optional and if provided will receive an error whenever emitted from the health check stream.

agonesSDK.health((error) => {
	console.error('error', error);
});

To mark the game server as ready to receive player connections, call the async method ready(). The result will be an empty object in this case.

let result = await agonesSDK.ready();

Similarly shutdown(), allocate(), setAnnotation(key, value) and setLabel(key, value) are async methods that perform an action and return an empty result.

To get details of the backing GameServer call the async method getGameServer(). The result will be an object representing GameServer defined in `sdk.proto` .

let result = await agonesSDK.getGameServer();

To get updates on the backing GameServer as they happen, call watchGameServer(callback, errorCallback). The callback will be called with a parameter matching the result of getGameServer(). The error callback is optional and if provided will receive an error whenever emitted from the watch stream.

agonesSDK.watchGameServer((result) => {
	console.log('watch', result);
}, (error) => {
	console.error('error', error);
});

To mark the game server as reserved for a period of time, call the async method reserve(seconds). The result will be an empty object.

For more information, please read the SDK Overview, check out agonesSDK.js and also look at the Node.js example .

Docs: Metrics

Thu, 03 Jan 2019 03:58:19 +0000

Agones controller exposes metrics via OpenCensus. OpenCensus is a single distribution of libraries that collect metrics and distributed traces from your services, we only use it for metrics but it will allow us to support multiple exporters in the future.

We choose to start with Prometheus as this is the most popular with Kubernetes but it is also compatible with Cloud Monitoring. If you need another exporter, check the list of supported exporters. It should be pretty straightforward to register a new one. (GitHub PRs are more than welcome.)

We plan to support multiple exporters in the future via environment variables and helm flags.

Backend integrations

Prometheus

If you are running a Prometheus instance you just need to ensure that metrics and kubernetes service discovery are enabled. (helm chart values agones.metrics.prometheusEnabled and agones.metrics.prometheusServiceDiscovery). This will automatically add annotations required by Prometheus to discover Agones metrics and start collecting them. (see example)

If your Prometheus metrics collection agent requires that you scrape from the pods directly(such as with Google Cloud Managed Prometheus), then the metrics ports for the controller and allocator will both be named http and exposed on 8080. In the case of the allocator, the port name and number can be overriden with the agones.allocator.serviceMetrics.http.portName and agones.allocator.serviceMetrics.http.port helm chart values.

Prometheus Operator

If you have Prometheus operator installed in your cluster, just enable ServiceMonitor installation in values:

agones:
  metrics:
    serviceMonitor:
      enabled: true

Google Cloud Managed Service for Prometheus

Google Cloud Managed Service for Prometheus is a fully managed multi-cloud solution for Prometheus. If you wish to use Managed Prometheus with Agones, follow the Google Cloud Managed Service for Prometheus installation steps.

Google Cloud Monitoring (formerly Stackdriver)

We support the OpenCensus Stackdriver exporter. In order to use it you should enable Cloud Monitoring API in Google Cloud Console. Follow the Google Cloud Monitoring installation steps to see your metrics in Cloud Monitoring.

Metrics available

Name	Description	Type
agones_gameservers_count	The number of gameservers per fleet and status	gauge
agones_gameserver_allocations_duration_seconds	The distribution of gameserver allocation requests latencies	histogram
agones_gameservers_total	The total of gameservers per fleet and status	counter
agones_gameserver_player_connected_total	The total number of players connected to gameservers (Only available when player tracking is enabled)	gauge
agones_gameserver_player_capacity_total	The available capacity for players on gameservers (Only available when player tracking is enabled)	gauge
agones_fleets_replicas_count	The number of replicas per fleet (total, desired, ready, reserved, allocated)	gauge
agones_fleet_counters	Aggregate Metrics for Counters within a Fleet, including total capacity and count values (Only available when Counters and Lists) are enabled)	gauge
agones_fleet_lists	Aggregate Metrics for Lists within a Fleet, including total capacity and List lengths (Only available when Counters and Lists) are enabled)	gauge
agones_fleet_autoscalers_able_to_scale	The fleet autoscaler can access the fleet to scale	gauge
agones_fleet_autoscalers_buffer_limits	The limits of buffer based fleet autoscalers (min, max)	gauge
agones_fleet_autoscalers_buffer_size	The buffer size of fleet autoscalers (count or percentage)	gauge
agones_fleet_autoscalers_current_replicas_count	The current replicas count as seen by autoscalers	gauge
agones_fleet_autoscalers_desired_replicas_count	The desired replicas count as seen by autoscalers	gauge
agones_fleet_autoscalers_limited	The fleet autoscaler is outside the limits set by MinReplicas and MaxReplicas.	gauge
agones_gameservers_node_count	The distribution of gameservers per node	histogram
agones_nodes_count	The count of nodes empty and with gameservers	gauge
agones_gameservers_state_duration	The distribution of gameserver state duration in seconds. Note: this metric could have some missing samples by design. Do not use the `_total` counter as the real value for state changes.	histogram
agones_k8s_client_http_request_total	The total of HTTP requests to the Kubernetes API by status code	counter
agones_k8s_client_http_request_duration_seconds	The distribution of HTTP requests latencies to the Kubernetes API by status code	histogram
agones_k8s_client_cache_list_total	The total number of list operations for client-go caches	counter
agones_k8s_client_cache_list_duration_seconds	Duration of a Kubernetes list API call in seconds	histogram
agones_k8s_client_cache_list_items	Count of items in a list from the Kubernetes API	histogram
agones_k8s_client_cache_watches_total	The total number of watch operations for client-go caches	counter
agones_k8s_client_cache_last_resource_version	Last resource version from the Kubernetes API	gauge
agones_k8s_client_workqueue_depth	Current depth of the work queue	gauge
agones_k8s_client_workqueue_latency_seconds	How long an item stays in the work queue	histogram
agones_k8s_client_workqueue_items_total	Total number of items added to the work queue	counter
agones_k8s_client_workqueue_work_duration_seconds	How long processing an item from the work queue takes	histogram
agones_k8s_client_workqueue_retries_total	Total number of items retried to the work queue	counter
agones_k8s_client_workqueue_longest_running_processor	How long the longest running workqueue processor has been running in microseconds	gauge
agones_k8s_client_workqueue_unfinished_work_seconds	How long unfinished work has been sitting in the workqueue in seconds	gauge

Dropping Metric Labels

When a Fleet or FleetAutoscaler is deleted from the system, Agones will automatically clear metrics that utilise their name as a label from the exported metrics, so the metrics exported do not continuously grow in size over the lifecycle of the Agones installation.

Dashboard

Grafana Dashboards

We provide a set of useful Grafana dashboards to monitor Agones workload, they are located under the grafana folder :

Agones Autoscalers allows you to monitor your current autoscalers replicas request as well as fleet replicas allocation and readyness statuses. You can only select one autoscaler at the time using the provided dropdown.
Agones GameServers displays your current game servers workload status (allocations, game servers statuses, fleets replicas) with optional fleet name filtering.
Agones GameServer Allocations displays Agones gameservers allocations rates and counts per fleet.
Agones Allocator Resource displays Agones Allocators CPU, memory usage and also some useful Golang runtime metrics.
Agones Status displays Agones controller health status.
Agones Controller Resource Usage displays Agones Controller CPU and memory usage and also some Golang runtime metrics.
Agones Controller go-client requests displays Agones Controller Kubernetes API consumption.
Agones Controller go-client caches displays Agones Controller Kubernetes Watches/Lists operations used.
Agones Controller go-client workqueues displays Agones Controller workqueue processing time and rates.
Agones Controller API Server requests displays your current API server request rate, errors rate and request latencies with optional CustomResourceDefinition filtering by Types: fleets, gameserversets, gameservers, gameserverallocations.

Dashboard screenshots :

Note

You can import our dashboards by copying the json content from each config map into your own instance of Grafana (+ > Create > Import > Or paste json) or follow the installation guide.

Installation

When operating a live multiplayer game you will need to observe performances, resource usage and availability to learn more about your system. This guide will explain how you can setup Prometheus and Grafana into your own Kubernetes cluster to monitor your Agones workload.

Before attemping this guide you should make sure you have kubectl and helm installed and configured to reach your kubernetes cluster.

Prometheus installation

Prometheus is an open source monitoring solution, we will use it to store Agones controller metrics and query back the data.

Let’s install Prometheus using the Prometheus Community Kubernetes Helm Charts repository.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm upgrade --install --wait prom prometheus-community/prometheus --namespace metrics --create-namespace \
    --set server.global.scrape_interval=30s \
    --set server.persistentVolume.enabled=true \
    --set server.persistentVolume.size=64Gi \
    -f ./build/prometheus.yaml

For resiliency it is recommended to run Prometheus on a dedicated node which is separate from nodes where Game Servers are scheduled. If you use the above command, with our prometheus.yaml to set up Prometheus, it will schedule Prometheus pods on nodes tainted with agones.dev/agones-metrics=true:NoExecute and labeled with agones.dev/agones-metrics=true if available.

As an example, to set up a dedicated node pool for Prometheus on GKE, run the following command before installing Prometheus. Alternatively you can taint and label nodes manually.

gcloud container node-pools create agones-metrics --cluster=... --zone=... \
  --node-taints agones.dev/agones-metrics=true:NoExecute \
  --node-labels agones.dev/agones-metrics=true \
  --num-nodes=1 \
  --machine-type=e2-standard-4

By default we will disable the push gateway (we don’t need it for Agones) and other exporters.

The helm chart supports nodeSelector, affinity and toleration, you can use them to schedule Prometheus deployments on an isolated node(s) to have an homogeneous game servers workload.

This will install a Prometheus Server in your current cluster with Persistent Volume Claim (Deactivated for Minikube and Kind) for storing and querying time series, it will automatically start collecting metrics from Agones Controller.

Finally, to access Prometheus metrics, rules and alerts explorer use

kubectl port-forward deployments/prom-prometheus-server 9090 -n metrics

Now you can access the prometheus dashboard http://localhost:9090.

On the landing page you can start exploring metrics by creating queries. You can also verify what targets Prometheus currently monitors (Header Status > Targets), you should see Agones controller pod in the kubernetes-pods section.

Note

Metrics will be first registered when you will start using Agones.

Now let’s install some Grafana dashboards.

Grafana installation

Grafana is a open source time series analytics platform which supports Prometheus data source. We can also easily import pre-built dashboards.

First we will install Agones dashboard as config maps in our cluster.

kubectl apply -f ./build/grafana/

Now we can install the Grafana Community Kubernetes Helm Charts from their repository. (Replace <your-admin-password> with the admin password of your choice)

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

helm upgrade --install --wait grafana grafana/grafana --namespace metrics \
  --set adminPassword=<your-admin-password> -f ./build/grafana.yaml

This will install Grafana with our prepopulated dashboards and prometheus datasource previously installed

Finally to access dashboards run

kubectl port-forward deployments/grafana 3000 -n metrics

Open a web browser to http://localhost:3000, you should see Agones dashboards after login as admin.

Google Cloud Managed Service for Prometheus installation

To collect Agones metrics using Managed Prometheus:

Follow the instructions to enable managed collection for a GKE cluster or non-GKE cluster.
Configure Managed Prometheus to scrape Agones by creating a PodMonitoring resource:

kubectl apply -n agones-system -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/build/prometheus-google-managed.yaml

Confirm that you can see Prometheus metrics in Cloud Monitoring. If that’s all you need, you can stop here.

To install Grafana using a Managed Prometheus backend:

Complete the Before you begin. To align with the Agones Grafana installation, we’ll be installing in the metrics namespace, which you’ll need to create.
- If your cluster has Workload Identity enabled, which is enabled on GKE Autopilot by default, follow Configure a service account for Workload Identity to ensure that you have appropriately authorized the default Kubernetes service account in the metrics namespace.
Install the Standalone Prometheus frontend UI in the metrics namespace - this will act as your authentication proxy for PromQL queries.
Install Grafana as above, using -f ./build/grafana-frontend.yaml instead of -f ./build/grafana.yaml.

Google Cloud Monitoring installation

In order to use Google Cloud Monitoring you must enable the Monitoring API in the Google Cloud Console. The Cloud Monitoring exporter uses a strategy called Application Default Credentials (ADC) to find your application’s credentials. Details can be found in Setting Up Authentication for Server to Server Production Applications.

You need to grant all the necessary permissions to the users (see Access Control Guide). The predefined role Monitoring Metric Writer contains those permissions. Use the following command to assign the role to your default service account.

gcloud projects add-iam-policy-binding [PROJECT_ID] --member serviceAccount:[PROJECT_NUMBER]-compute@developer.gserviceaccount.com --role roles/monitoring.metricWriter

Note

Cloud Operations for GKE (including Cloud Monitoring) is enabled by default on GKE clusters, however you can follow this guide if it is currently disabled in your GKE cluster.

Before proceeding, ensure you have created a metrics node pool as mentioned in the Google Cloud installation guide.

The default metrics exporter installed with Agones is Prometheus. If you are using the Helm installation, you can install or upgrade Agones to use Cloud Monitoring, using the following chart parameters:

helm upgrade --install --wait --set agones.metrics.stackdriverEnabled=true --set agones.metrics.prometheusEnabled=false --set agones.metrics.prometheusServiceDiscovery=false my-release-name agones/agones --namespace=agones-system

Note

If you are using the YAML installation, follow the instructions on the page to change the above parameters by using helm to generate a custom YAML file locally.

With this configuration only the Cloud Monitoring exporter would be used instead of Prometheus exporter.

Using Cloud Monitoring with Workload Identity

If you would like to enable Cloud Monitoring in conjunction with Workload Identity, there are a few extra steps you need to follow:

When setting up the Google service account following the instructions for Authenticating to Google Cloud, create two IAM policy bindings, one for serviceAccount:PROJECT_ID.svc.id.goog[agones-system/agones-controller] and one for serviceAccount:PROJECT_ID.svc.id.goog[agones-system/agones-allocator].
Pass parameters to helm when installing Agones to add annotations to the agones-controller and agones-allocator Kubernetes service accounts:

helm install my-release --namespace agones-system --create-namespace agones/agones --set agones.metrics.stackdriverEnabled=true --set agones.metrics.prometheusEnabled=false --set agones.metrics.prometheusServiceDiscovery=false --set agones.serviceaccount.allocator.annotations."iam\.gke\.io/gcp-service-account"="GSA_NAME@PROJECT_ID\.iam\.gserviceaccount\.com" --set agones.serviceaccount.allocator.labels."iam\.gke\.io/gcp-service-account"="GSA_NAME@PROJECT_ID\.iam\.gserviceaccount\.com" --set agones.serviceaccount.controller.annotations."iam\.gke\.io/gcp-service-account"="GSA_NAME@PROJECT_ID\.iam\.gserviceaccount\.com"

To verify that metrics are being sent to Cloud Monitoring, create a Fleet or a Gameserver and look for the metrics to show up in the Cloud Monitoring dashboard. Navigate to the Metrics explorer and search for metrics with the prefix agones/. Select a metric and look for data to be plotted in the graph to the right.

An example of a custom dashboard is:

Currently there exists only manual way of configuring Cloud Monitoring Dashboard. So it is up to you to set an Alignment Period (minimal is 1 minute), GroupBy, Filter parameters and other graph settings.

Troubleshooting

If you can’t see Agones metrics you should have a look at the controller logs for connection errors. Also ensure that your cluster has the necessary credentials to interact with Cloud Monitoring. You can configure stackdriverProjectID manually, if the automatic discovery is not working.

Permissions problem example from controller logs:

Failed to export to Stackdriver: rpc error: code = PermissionDenied desc = Permission monitoring.metricDescriptors.create denied (or the resource may not exist).

If you receive this error, ensure your service account has the role or corresponding permissions mentioned above.

Docs: Access Agones via the Kubernetes API

Thu, 03 Jan 2019 01:20:41 +0000

Installing Agones creates several Custom Resource Definitions (CRD), which can be accessed and manipulated through the Kubernetes API.

The detailed list of Agones CRDs with their parameters could be found here - Agones CRD API Reference.

Kubernetes has multiple client libraries, however, at time of writing, only the Go and Python clients are documented to support accessing CRDs.

This can be found in the Accessing a custom resource section of the Kubernetes documentation.

At this time, we recommend interacting with Agones through the Go client that has been generated in this repository, but other methods may also work as well.

Go Client

Kubernetes Go Client tooling generates a Client for Agones that we can use to interact with the Agones installation on our Kubernetes cluster.

Authentication

This client uses the same authentication mechanisms as the Kubernetes API.

If you plan to run your code in the same cluster as the Agones install, have a look at the in cluster configuration example from the Kubernetes Client.

If you plan to run your code outside the Kubernetes cluster as your Agones install, look at the out of cluster configuration example from the Kubernetes client.

Example

The following is an example of a in-cluster configuration, that creates a Clientset for Agones and then creates a GameServer.

A full example code is available in the example folder .

package main

import (
	"fmt"
	"context"

	agonesv1 "agones.dev/agones/pkg/apis/agones/v1"
	"agones.dev/agones/pkg/client/clientset/versioned"
	"agones.dev/agones/pkg/util/runtime"
	corev1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/rest"
)

func main() {
	config, err := rest.InClusterConfig()
	logger := runtime.NewLoggerWithSource("main")
	if err != nil {
		logger.WithError(err).Fatal("Could not create in cluster config")
	}

	// Access to standard Kubernetes resources through the Kubernetes Clientset
	// We don't actually need this for this example, but it's just here for
	// illustrative purposes
	kubeClient, err := kubernetes.NewForConfig(config)
	if err != nil {
		logger.WithError(err).Fatal("Could not create the kubernetes clientset")
	}

	// Access to the Agones resources through the Agones Clientset
	// Note that we reuse the same config as we used for the Kubernetes Clientset
	agonesClient, err := versioned.NewForConfig(config)
	if err != nil {
		logger.WithError(err).Fatal("Could not create the agones api clientset")
	}

	// Create a GameServer
	gs := &agonesv1.GameServer{ObjectMeta: metav1.ObjectMeta{GenerateName: "simple-game-server", Namespace: "default"},
		Spec: agonesv1.GameServerSpec{
			Container: "simple-game-server",
			Ports: []agonesv1.GameServerPort{{
				ContainerPort: 7654,
				HostPort:      7654,
				Name:          "gameport",
				PortPolicy:    agonesv1.Static,
				Protocol:      corev1.ProtocolUDP,
			}},
			Template: corev1.PodTemplateSpec{
				Spec: corev1.PodSpec{
					Containers: []corev1.Container{{Name: "simple-game-server", Image: "us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32"}},
				},
			},
		},
	}
	newGS, err := agonesClient.AgonesV1().GameServers("default").Create(context.TODO(), gs, metav1.CreateOptions{})
	if err != nil {
		panic(err)
	}

	fmt.Printf("New game servers' name is: %s", newGS.ObjectMeta.Name)
}

In order to create GS using provided example, you can run it as a Kubernetes Job:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.40.0/examples/crd-client/create-gs.yaml --namespace agones-system
kubectl get pods --namespace agones-system

NAME                                 READY   STATUS      RESTARTS   AGE
create-gs-6wz86-7qsm5                0/1     Completed   0          6s

kubectl logs create-gs-6wz86-7qsm5  --namespace agones-system

{"message":"\u0026{0xc0001dde00 default}","severity":"info","source":"main","time":"2020-04-21T11:14:00.477576428Z"}
{"message":"New GameServer name is: helm-test-server-fxfgg","severity":"info","time":"2020-04-21T11:14:00.516024697Z"}

You have just created a GameServer using Kubernetes Go Client.

Best Practice: Using Informers and Listers

Almost all Kubernetes’ controllers and custom controllers utilise Informers and Listers to reduce the load on the Kubernetes’s control plane.

Repetitive, direct access of the Kubernetes control plane API can significantly reduce the performance of the cluster – and Informers and Listers help resolving that issue.

Informers and Listers reduce the load on the Kubernetes control plane by creating, using and maintaining an eventually consistent an in-memory cache. This can be watched and also queried with zero cost, since it will only read against its in-memory model of the Kubernetes resources.

Informer’s role and Lister’s role are different.

An Informer is the mechanism for watching a Kubernetes object’s event, such that when a Kubernetes object changes(e.g. CREATE,UPDATE,DELETE), the Informer is informed, and can execute a callback with the relevant object as an argument.

This can be very useful for building event based systems against the Kubernetes API.

A Lister is the mechanism for querying Kubernetes object’s against the client side in-memory cache. Since the Lister stores objects in an in-memory cache, queries against a come at practically no cost.

Of course, Agones itself also uses Informers and Listers in its codebase.

Example

The following is an example of Informers and Listers, that show the GameServer’s name & status & IPs in the Kubernetes cluster.

package main

import (
	"context"
	"time"

	"agones.dev/agones/pkg/client/clientset/versioned"
	"agones.dev/agones/pkg/client/informers/externalversions"
	"agones.dev/agones/pkg/util/runtime"
	"k8s.io/apimachinery/pkg/labels"
	"k8s.io/client-go/informers"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/cache"
)

func main() {
	config, err := rest.InClusterConfig()
	logger := runtime.NewLoggerWithSource("main")
	if err != nil {
		logger.WithError(err).Fatal("Could not create in cluster config")
	}
	kubeClient, err := kubernetes.NewForConfig(config)
	agonesClient, err := versioned.NewForConfig(config)
	if err != nil {
		logger.WithError(err).Fatal("Could not create the agones api clientset")
	}

	// Create InformerFactory which create the informer
	informerFactory := informers.NewSharedInformerFactory(kubeClient, time.Second*30)
	agonesInformerFactory := externalversions.NewSharedInformerFactory(agonesClient, time.Second*30)

	// Create Pod informer by informerFactory
	podInformer := informerFactory.Core().V1().Pods()

	// Create GameServer informer by informerFactory
	gameServers := agonesInformerFactory.Agones().V1().GameServers()
	gsInformer := gameServers.Informer()

	// Add EventHandler to informer
	// When the object's event happens, the function will be called
	// For example, when the pod is added, 'AddFunc' will be called and put out the "Pod Added"
	podInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
		AddFunc:    func(new interface{}) { logger.Infof("Pod Added") },
		UpdateFunc: func(old, new interface{}) { logger.Infof("Pod Updated") },
		DeleteFunc: func(old interface{}) { logger.Infof("Pod Deleted") },
	})
	gsInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
		AddFunc:    func(new interface{}) { logger.Infof("GameServer Added") },
		UpdateFunc: func(old, new interface{}) { logger.Infof("GameServer Updated") },
		DeleteFunc: func(old interface{}) { logger.Infof("GameServer Deleted") },
	})

	ctx := context.Background()

	// Start Go routines for informer
	informerFactory.Start(ctx.Done())
	agonesInformerFactory.Start(ctx.Done())
	// Wait until finish caching with List API
	informerFactory.WaitForCacheSync(ctx.Done())
	agonesInformerFactory.WaitForCacheSync(ctx.Done())

	// Create Lister which can list objects from the in-memory-cache
	podLister := podInformer.Lister()
	gsLister := gameServers.Lister()

	for {
		// Get List objects of Pods from Pod Lister
		p := podLister.Pods("default")
		// Get List objects of GameServers from GameServer Lister
		gs, err := gsLister.List(labels.Everything())
		if err != nil {
			panic(err)
		}
		// Show GameServer's name & status & IPs
		for _, g := range gs {
			a, err := p.Get(g.GetName())
			if err != nil {
				panic(err)
			}
			logger.Infof("------------------------------")
			logger.Infof("Name: %s", g.GetName())
			logger.Infof("Status: %s", g.Status.State)
			logger.Infof("External IP: %s", g.Status.Address)
			logger.Infof("Internal IP: %s", a.Status.PodIP)
		}
		time.Sleep(time.Second * 25)
	}
}

You can list GameServer’s name and status and IPs using Kubernetes Informers and Listers.

Direct Access to the REST API via Kubectl

If there isn’t a client written in your preferred language, it is always possible to communicate directly with Kubernetes API to interact with Agones.

The Kubernetes API can be authenticated and exposed locally through the kubectl proxy

For example:

kubectl proxy &

Starting to serve on 127.0.0.1:8001

List all Agones endpoints

curl http://localhost:8001/apis | grep agones -A 5 -B 5

{
    "name": "agones.dev",
    "versions": [
    {
        "groupVersion": "agones.dev/v1",
        "version": "v1"
    }
    ],
    "preferredVersion": {
    "groupVersion": "agones.dev/v1",
    "version": "v1"
    },
    "serverAddressByClientCIDRs": null
}

List Agones resources

curl http://localhost:8001/apis/agones.dev/v1

{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "agones.dev/v1",
  "resources": [
    {
      "name": "gameservers",
      "singularName": "gameserver",
      "namespaced": true,
      "kind": "GameServer",
      "verbs": [
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "create",
        "update",
        "watch"
      ],
      "shortNames": [
        "gs"
      ]
    }
  ]
}

List all gameservers in the default namespace

curl http://localhost:8001/apis/agones.dev/v1/namespaces/default/gameservers

{
    "apiVersion": "agones.dev/v1",
    "items": [
        {
            "apiVersion": "agones.dev/v1",
            "kind": "GameServer",
            "metadata": {
                "annotations": {
                    "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"agones.dev/v1\",\"kind\":\"GameServer\",\"metadata\":{\"annotations\":{},\"name\":\"simple-game-server\",\"namespace\":\"default\"},\"spec\":{\"containerPort\":7654,\"hostPort\":7777,\"portPolicy\":\"static\",\"template\":{\"spec\":{\"containers\":[{\"image\":\"us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32\",\"name\":\"simple-game-server\"}]}}}}\n"
                },
                "clusterName": "",
                "creationTimestamp": "2018-03-02T21:41:05Z",
                "finalizers": [
                    "agones.dev"
                ],
                "generation": 0,
                "name": "simple-game-server",
                "namespace": "default",
                "resourceVersion": "760",
                "selfLink": "/apis/agones.dev/v1/namespaces/default/gameservers/simple-game-server",
                "uid": "692beea6-1e62-11e8-beb2-080027637781"
            },
            "spec": {
                "PortPolicy": "Static",
                "container": "simple-game-server",
                "containerPort": 7654,
                "health": {
                    "failureThreshold": 3,
                    "initialDelaySeconds": 5,
                    "periodSeconds": 5
                },
                "hostPort": 7777,
                "protocol": "UDP",
                "template": {
                    "metadata": {
                        "creationTimestamp": null
                    },
                    "spec": {
                        "containers": [
                            {
                                "image": "us-docker.pkg.dev/agones-images/examples/simple-game-server:0.32",
                                "name": "simple-game-server",
                                "resources": {}
                            }
                        ]
                    }
                }
            },
            "status": {
                "address": "192.168.99.100",
                "nodeName": "agones",
                "port": 7777,
                "state": "Ready"
            }
        }
    ],
    "kind": "GameServerList",
    "metadata": {
        "continue": "",
        "resourceVersion": "1062",
        "selfLink": "/apis/agones.dev/v1/namespaces/default/gameservers"
    }
}

Allocate a gameserver from a fleet named ‘simple-game-server’, with GameServerAllocation

curl -d '{"apiVersion":"allocation.agones.dev/v1","kind":"GameServerAllocation","spec":{"required":{"matchLabels":{"agones.dev/fleet":"simple-game-server"}}}}' -H "Content-Type: application/json" -X POST http://localhost:8001/apis/allocation.agones.dev/v1/namespaces/default/gameserverallocations

{
    "kind": "GameServerAllocation",
    "apiVersion": "allocation.agones.dev/v1",
    "metadata": {
        "name": "simple-game-server-v6jwb-cmdcv",
        "namespace": "default",
        "creationTimestamp": "2019-07-03T17:19:47Z"
    },
    "spec": {
        "multiClusterSetting": {
            "policySelector": {}
        },
        "required": {
            "matchLabels": {
                "agones.dev/fleet": "simple-game-server"
            }
        },
        "scheduling": "Packed",
        "metadata": {}
    },
    "status": {
        "state": "Allocated",
        "gameServerName": "simple-game-server-v6jwb-cmdcv",
        "ports": [
            {
                "name": "default",
                "port": 7445
            }
        ],
        "address": "34.94.118.237",
        "nodeName": "gke-test-cluster-default-f11755a7-5km3"
    }
}

You may wish to review the Agones Kubernetes API for the full data structure reference.

The Kubernetes API Concepts section may also provide the more details on the API conventions that are used in the Kubernetes API.

Next Steps

Learn how to use Allocator Service for single and multi-cluster Allocation.

Docs: Rust Game Server Client SDK

Wed, 02 Jan 2019 10:17:57 +0000

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Area	Action	Implemented
Lifecycle	Ready	✔️
Lifecycle	Health	✔️
Lifecycle	Reserve	✔️
Lifecycle	Allocate	✔️
Lifecycle	Shutdown	✔️
Configuration	GameServer	✔️
Configuration	Watch	✔️
Metadata	SetAnnotation	✔️
Metadata	SetLabel	✔️
Counters	GetCounterCount	❌
Counters	SetCounterCount	❌
Counters	IncrementCounter	❌
Counters	DecrementCounter	❌
Counters	SetCounterCapacity	❌
Counters	GetCounterCapacity	❌
Lists	AppendListValue	❌
Lists	DeleteListValue	❌
Lists	SetListCapacity	❌
Lists	GetListCapacity	❌
Lists	ListContains	❌
Lists	GetListLength	❌
Lists	GetListValues	❌
Player Tracking	GetConnectedPlayers	✔️
Player Tracking	GetPlayerCapacity	✔️
Player Tracking	GetPlayerCount	✔️
Player Tracking	IsPlayerConnected	✔️
Player Tracking	PlayerConnect	✔️
Player Tracking	PlayerDisconnect	✔️
Player Tracking	SetPlayerCapacity	✔️

Prerequisites

Rust >= 1.50

Usage

Add this crate to dependencies section in your Cargo.toml.

Also note that the SDK is async only, so you will need an async runtime to execute the futures exposed by the SDK. It is recommended to use tokio as the SDK already depends on tokio due to its choice of gRPC library, tonic.

[dependencies]
agones = "1.34.0"
tokio = { version = "1.32.0", features = ["macros", "sync"] }

To begin working with the SDK, create an instance of it.

use std::time::Duration;

#[tokio::main]
async fn main() {
    let mut sdk = agones::Sdk::new(None /* default port */, None /* keep_alive */)
        .await
        .expect("failed to connect to SDK server");
}

To send health checks, call sdk.health_check, which will return a tokio::sync::mpsc::Sender::<()> which will send a health check every time a message is posted to the channel.

let health = sdk.health_check();
if health.send(()).await.is_err() {
    eprintln!("the health receiver was closed");
}

To mark the game session as ready call sdk.ready().

sdk.ready().await?;

To mark the game server as reserved for a period of time, call sdk.reserve(duration).

sdk.reserve(Duration::new(5, 0)).await?;

To mark that the game session is completed and the game server should be shut down call sdk.shutdown().

if let Err(e) = sdk.shutdown().await {
    eprintln!("Could not run Shutdown: {}", e);
}

To set a Label on the backing GameServer call sdk.set_label(key, value).

sdk.set_label("test-label", "test-value").await?;

To set an Annotation on the backing GameServer call sdk.set_annotation(key, value).

sdk.set_annotation("test-annotation", "test value").await?;

To get details of the backing GameServer call sdk.get_gameserver().

The function will return an instance of agones::types::GameServer including GameServer configuration info.

let gameserver = sdk.get_gameserver().await?;

To get updates on the backing GameServer as they happen, call sdk.watch_gameserver.

This will stream updates and endlessly until the stream is closed, so it is recommended to push this into its own async task.

let _watch = {
    // We need to clone the SDK as we are moving it to another task
    let mut watch_client = sdk.clone();
    // We use a simple oneshot to signal to the task when we want it to shutdown
    // and stop watching the gameserver update stream
    let (tx, mut rx) = tokio::sync::oneshot::channel::<()>();

    tokio::task::spawn(async move {
        println!("Starting to watch GameServer updates...");
        match watch_client.watch_gameserver().await {
            Err(e) => eprintln!("Failed to watch for GameServer updates: {}", e),
            Ok(mut stream) => loop {
                tokio::select! {
                    // We've received a new update, or the stream is shutting down
                    gs = stream.message() => {
                        match gs {
                            Ok(Some(gs)) => {
                                println!("GameServer Update, name: {}", gs.object_meta.unwrap().name);
                                println!("GameServer Update, state: {}", gs.status.unwrap().state);
                            }
                            Ok(None) => {
                                println!("Server closed the GameServer watch stream");
                                break;
                            }
                            Err(e) => {
                                eprintln!("GameServer Update stream encountered an error: {}", e);
                            }
                        }

                    }
                    // The watch is being dropped so we're going to shutdown the task
                    // and the watch stream
                    _ = &mut rx => {
                        println!("Shutting down GameServer watch loop");
                        break;
                    }
                }
            },
        }
    });

    tx
};

For more information, please read the SDK Overview, check out agones sdk implementation and also look at the Rust example .

Docs: High Density GameServers

Tue, 31 Aug 2021 00:00:00 +0000

Depending on the setup and resource requirements of your game server process, sometimes it can be a more economical use of resources to run multiple concurrent game sessions from within a single GameServer instance.

The tradeoff here is that this requires more management on behalf of the integrated game server process and external systems, since it works around the common Kubernetes and/or Agones container lifecycle.

Utilising the new allocation gameServerState filter as well as the existing ability to edit the GameServer labels at both allocation time, and from within the game server process, via the SDK, means Agones is able to atomically remove a GameServer from the list of potentially allocatable GameServers at allocation time, and then return it back into the pool of allocatable GameServers if and when the game server process deems that is has room to host another game session.

Info

To watch for Allocation events, there is the initial GameServer.status.state change from Ready to Allocated, but it is also useful to know that the value of GameServer.metadata.annotations["agones.dev/last-allocated"] will change as it is set by Agones with each allocation with the current timestamp, regardless of if there is a state change or not.

Example `GameServerAllocation`

The below Allocation will first attempt to find a GameServer from the Fleet simple-udp that is already Allocated and also has the label agones.dev/sdk-gs-session-ready with the value of true.

The above condition indicates that the matching game server process behind the matched GameServer record is able to accept another game session at this time.

If an Allocated GameServer does not exist with the desired labels, then use the next selector to allocate a Ready GameServer from the simple-udp Fleet.

Whichever condition is met, once allocation is made against a GameServer, its label of agones.dev/sdk-gs-session-ready will be set to the value of false and it will no longer match the first selector, thereby removing it from any future allocations with the below schema.

It will then be up to the game server process to decide on if and when it is appropriate to set the agones.dev/sdk-gs-session-ready value back to true, thereby indicating that it can accept another concurrent gameplay session.

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        agones.dev/fleet: simple-udp
        agones.dev/sdk-gs-session-ready: "true" # this is important
      gameServerState: Allocated # new state filter: allocate from Allocated servers
    - matchLabels:
        agones.dev/fleet: simple-udp
      gameServerState: Ready # Allocate out of the Ready Pool (which would be default, so backward compatible)
  metadata:
    labels:
      agones.dev/sdk-gs-session-ready: "false" # this removes it from the pool

Info

It’s important to note that the labels that the GameServer process use to add itself back into the pool of allocatable instances, must start with the prefix agones.dev/sdk-, since only labels that have this prefix are available to be [updated from the SDK][sdk].

Consistency

Agones, and Kubernetes itself are built as eventually consistent, self-healing systems. To that end, it is worth noting that there may be minor delays between each of the operations in the above flow. For example, depending on the cluster load, it may take up to a second for an SDK driven label change on a GameServer record to be visible to the Agones allocation system. We recommend building your integrations with Agones with this in mind.

Next Steps

View the details about using the SDK to set labels on the GameServer.
Check all the options available on GameServerAllocation.

Docs: Allocating based on GameServer Player Capacity

Tue, 31 Aug 2021 00:00:00 +0000

Counters and Lists will eventually replace the Alpha functionality of Player Tracking, which will subsequently be removed from Agones.

If you are currently using this Alpha feature, we would love for you to test (and ideally migrate to!) this new functionality to Counters and Lists to ensure it meet all your needs.

This document will be updated to utilise Counters and Lists in the near future.

Warning

The Player Tracking and Allocation Player Filter features are currently Alpha, not enabled by default, and may change in the future.

Use the FeatureGates PlayerTracking, and PlayerAllocationFilter to enable and test these features.

See the Feature Gate documentation for details on how to enable features.

Using this approach, we are able to be able to make a request that is akin to: “Find me a GameServer that is already allocated, with room for n number of players, and if one is not available, allocate me a Ready GameServer”.

Common applications of this type of allocation are Lobby servers where players await matchmaking, or a persistent world server where players connect and disconnect from a large map.

Example `GameServerAllocation`

The below allocation will attempt to find an already Allocated GameServer from the Fleet “lobby” with room for 10 to 15 players, and if it cannot find one, will allocate a Ready one from the same Fleet.

apiVersion: "allocation.agones.dev/v1"
kind: GameServerAllocation
spec:
  selectors:
    - matchLabels:
        agones.dev/fleet: lobby
      gameServerState: Allocated
      players:
        minAvailable: 10
        maxAvailable: 15
    - matchLabels:
        agones.dev/fleet: lobby

Note

We recommend doing an extra check when players connect to a GameServer that there is the expected player capacity on the GameServer as there can be a small delay between a player connecting and it being reported to Agones.

Next Steps

Have a look at all commands the Client SDK provides.
Check all the options available on GameServerAllocation.
If you aren’t familiar with the term Pod, this should provide a reference.

Docs: Confirming Agones Installation

Mon, 01 Jan 0001 00:00:00 +0000

To confirm Agones is up and running, run the following command:

kubectl describe --namespace agones-system pods

It should describe six pods created in the agones-system namespace, with no error messages or status. All Conditions sections should look like this:

Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True

All this pods should be in a RUNNING state:

kubectl get pods --namespace agones-system

NAME                                 READY   STATUS    RESTARTS   AGE
agones-allocator-5c988b7b8d-cgtbs    1/1     Running   0          8m47s
agones-allocator-5c988b7b8d-hhhr5    1/1     Running   0          8m47s
agones-allocator-5c988b7b8d-pv577    1/1     Running   0          8m47s
agones-controller-7db45966db-56l66   1/1     Running   0          8m44s
agones-ping-84c64f6c9d-bdlzh         1/1     Running   0          8m37s
agones-ping-84c64f6c9d-sjgzz         1/1     Running   0          8m47s

That’s it!

Now with Agones installed, you can utilise its Custom Resource Definitions to create resources of type GameServer, Fleet and more!

What’s next

Go through the Create a Game Server Quickstart

Docs: Upgrading Agones and Kubernetes

Fri, 16 Aug 2019 00:19:19 +0000

Note

Whichever approach you take to upgrading Agones, make sure to test it in your development environment before applying it to production.

Upgrading Agones

The following are strategies for safely upgrading Agones from one version to another. They may require adjustment to your particular game architecture but should provide a solid foundation for updating Agones safely.

The recommended approach is to use multiple clusters, such that the upgrade can be tested gradually with production load and easily rolled back if the need arises.

Warning

Changing Feature Gates within your Agones install can constitute an “upgrade” as it may create or remove functionality in the Agones installation that may not be forward or backward compatible with installed resources in an existing installation.

Upgrading Agones: Multiple Clusters

We essentially want to transition our GameServer allocations from a cluster with the old version of Agones, to a cluster with the upgraded version of Agones while ensuring nothing surprising happens during this process.

This also allows easy rollback to the previous infrastructure that we already know to be working in production, with minimal interruptions to player experience.

The following are steps to implement this:

Create a new cluster of the same size or smaller as the current cluster.
Install the new version of Agones on the new cluster.
Deploy the same set of Fleets, GameServers and FleetAutoscalers from the old cluster into the new cluster.
With your matchmaker, start sending a small percentage of your matched players’ game sessions to the new cluster.
Assuming everything is working successfully on the new cluster, slowly increase the percentage of matched sessions to the new cluster, until you reach 100%.
Once you are comfortable with the stability of the new cluster with the new Agones version, shut down the old cluster.
Congratulations - you have now upgraded to a new version of Agones! 👍

Upgrading Agones: Single Cluster

If you are upgrading a single cluster, we recommend creating a maintenance window, in which your game goes offline for the period of your upgrade, as there will be a short period in which Agones will be non-responsive during the upgrade.

Installation with install.yaml

If you installed Agones with install.yaml, then you will need to delete the previous installation of Agones before upgrading to the new version, as we need to remove all of Agones before installing the new version.

Start your maintenance window.
Delete the current set of Fleets, GameServers and FleetAutoscalers in your cluster.
Make sure to delete the same version of Agones that was previously installed, for example: kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/<old-release-version>/install/yaml/install.yaml
Install Agones with install.yaml.
Deploy the same set of Fleets, GameServers and FleetAutoscalers back into the cluster.
Run any other tests to ensure the Agones installation is working as expected.
Close your maintenance window.
Congratulations - you have now upgraded to a new version of Agones! 👍

Installation with Helm

Helm features capabilities for upgrading to newer versions of Agones without having to uninstall Agones completely.

For details on how to use Helm for upgrades, see the helm upgrade documentation.

Given the above, the steps for upgrade are simpler:

Start your maintenance window.
Delete the current set of Fleets, GameServers and FleetAutoscalers in your cluster.
Run helm upgrade with the appropriate arguments, such a --version, for your specific upgrade
Deploy the same set of Fleets, GameServers and FleetAutoscalers back into the cluster.
Run any other tests to ensure the Agones installation is working as expected.
Close your maintenance window.
Congratulations - you have now upgraded to a new version of Agones! 👍

Upgrading Kubernetes

The following are strategies for safely upgrading the underlying Kubernetes cluster from one version to another. They may require adjustment to your particular game architecture but should provide a solid foundation for updating your cluster safely.

The recommended approach is to use multiple clusters, such that the upgrade can be tested gradually with production load and easily rolled back if the need arises.

Agones has multiple supported Kubernetes versions for each version. You can stick with a minor Kubernetes version until it is not supported by Agones, but it is recommended to do supported minor (e.g. 1.12.1 ➡ 1.13.2) Kubernetes version upgrades at the same time as a matching Agones upgrades.

Patch upgrades (e.g. 1.12.1 ➡ 1.12.3) within the same minor version of Kubernetes can be done at any time.

Multiple Clusters

This process is very similar to the Upgrading Agones: Multiple Clusters approach above.

We essentially want to transition our GameServer allocations from a cluster with the old version of Kubernetes, to a cluster with the upgraded version of Kubernetes while ensuring nothing surprising happens during this process.

This also allows easy rollback to the previous infrastructure that we already know to be working in production, with minimal interruptions to player experience.

The following are steps to implement this:

Create a new cluster of the same size or smaller as the current cluster, with the new version of Kubernetes
Install the same version of Agones on the new cluster, as you have on the previous cluster.
Deploy the same set of Fleets and/or GameServers from the old cluster into the new cluster.
With your matchmaker, start sending a small percentage of your matched players’ game sessions to the new cluster.
Assuming everything is working successfully on the new cluster, slowly increase the percentage of matched sessions to the new cluster, until you reach 100%.
Once you are comfortable with the stability of the new cluster with the new Kubernetes version, shut down the old cluster.
Congratulations - you have now upgraded to a new version of Kubernetes! 👍

Single Cluster

Start your maintenance window.
Scale your Fleets down to 0 and/or delete your GameServers. This is a good safety measure so there aren’t race conditions between the Agones controller being recreated and GameServers being deleted doesn’t occur, and GameServers can end up stuck in erroneous states.
Start and complete your control plane upgrade(s).
Start and complete your node upgrades.
Scale your Fleets back up and/or recreate your GameServers.
Run any other tests to ensure the Agones installation is still working as expected.
Close your maintenance window.
Congratulations - you have now upgraded to a new version of Kubernetes! 👍

Agones – Agones Documentation

Docs: System Diagram

Agones Control Plane

agones-allocator

agones-controller

agones-extensions

agones-ping (not pictured)

Agones CRDs

Game Server Pod

Docs: Feature Stages

Supported Versions

Agones Features

Feature Gates

Note

Description of Stages

Alpha

Note

Beta

Note

Stable

Feature Stage Indicators

Custom Resource Definitions (CRDs)

New CRDs

New CRD attributes

Agones Game Server SDK

REST & gRPC APIs

Docs: Matchmaker requests a GameServer from a Fleet

Sample GameServerAllocation

Next Steps:

Docs: Unreal Engine Game Server Client Plugin

SDK Functionality

Download

Resources

Getting Started

Getting the Code

Using C++ (UE5/UE4)

Using Blueprints (UE5)

Using Blueprints (UE4)

Configuration Options

Unreal Hooks

Docs: Third Party Videos and Presentations

Presentations

Large-Scale Multiplayer Gaming on Kubernetes (Cloud Next ‘19)

Agones: Scaling Multiplayer Game Servers with Open Source (GDC 2019)

Intro to Agones: Scaling Multiplayer Game Servers with Kubernetes (Kubecon, 2018)

Google Cloud Next ‘18 London: Carl Dionne, Development Director at Ubisoft Montreal

Agones: Scaling Multiplayer Dedicated Game Servers with Kubernetes (Cloud Next ‘18)

Screencasts

Agones: How Do I Docker and Kubernetes? (Part 1 & Part 2)

Docs: Scheduling and Autoscaling

Cluster Autoscaler

Google Kubernetes Engine

Amazon Elastic Kubernetes Service

Azure Kubernetes Service

Fleet Autoscaling

Autoscaling Concepts

Allocation Scheduling

Pod Scheduling

Fleet Scale Down Strategy

Fleet Scheduling

Packed

Cluster Autoscaler

Allocation Scheduling Strategy

Pod Scheduling Strategy

Note

Fleet Scale Down Strategy

Distributed

Note

Cluster Autoscaler

Allocation Scheduling Strategy

Pod Scheduling Strategy

Fleet Scale Down Strategy

Docs: GameServer Specification

Note

Stable Network ID

GameServer State Diagram

Primary Address vs Addresses

Docs: Quickstart: Create a Game Server

Prerequisites

Objectives

`agones-allocator`

`agones-controller`

`agones-extensions`

`agones-ping` (not pictured)

Sample `GameServerAllocation`