Note: You are looking at documentation for an outdated Giant Swarm product. Visit docs.giantswarm.io for up-to-date documentation of our current products.

Last modified March 31, 2016

Service Definition (swarm.json) Reference

Giant Swarm services are defined using a JSON file format. Files containing Giant Swarm service definition are commonly called swarm.json.

To get a basic understanding of how Giant Swarm services are built, we recommend that you have looked at our Introduction, especially at the section called The Anatomy of a Giant Swarm Service.

Basic Structure

The basic structure of the service definition file is:

  • A service name (optional, but recommended)
  • A set of components

Simple Example

{
  "name": "simple_service",
  "components": {
    "webserver": {
      "image": "registry.giantswarm.io/myorg/webserver",
      "ports": 80,
      "domains": {
        "80": "mysite.example.com"
      },
      "links": [
        {
          "component": "database",
          "target_port": 3306
        }
      ]
    },
    "database": {
      "image": "mysql",
      "ports": 3306
    }
  }
}

The example above illustrates the basic structure by defining a service called simple_service. This service consists of two components called webserver and database. The webserver component uses a custom Docker image that has been deployed on Giant Swarm’s registry. Port 80 is exposed and made publicly accessible by mapping a domains entry to it. It links to the database component, which uses the official MySQL image, on TCP port 3306.

Complex example

This example makes use of all possible keys to illustrate their use.

{
  "name": "complex_service",
  "components": {
    "appserver": {
      "image": "registry.giantswarm.io/myorg/appserver:latest",
      "entrypoint": "/opt/bin/myprogram",
      "args": ["--some-args", "hello world"],
      "volumes": [
        {
          "path": "/var/data",
          "size": "2 GB"
        }
      ],
      "env": {
        "MODE": "development"
      },
      "links": [
        {
          "component": "datastore",
          "target_port": 6379,
          "alias": "redis"
        }
      ],
      "scale": {
        "min": 2,
        "max": 5,
        "placement": "one-per-machine"
      },
      "ports": [8000, 8080],
      "domains": {
        "8000": "www.$GIANTSWARM_LOGIN_NAME.gigantic.io",
        "8080": "admin.$GIANTSWARM_LOGIN_NAME.gigantic.io"
      },
      "signal-ready": true
    },
    "datastore": {
      "pod": "children",
      "expose": [
        {
          "component": "datastore/redis",
          "target_port": "6379",
          "port": "6379"
        }
      ]
    },
    "datastore/redis": {
      "image": "redis",
      "ports": 6379,
      "volumes": [
        {
          "path": "/var/data",
          "size": "2 GB"
        }
      ]
    },
    "datastore/redisbackup": {
      "image": "registry.giantswarm.io/myorg/redis-backup:latest",
      "volumes": [
        {
          "volumes-from": "datastore/redis"
        }
      ]
    }
  }
}

All configuration directives shown in the examples are explained below, starting with those on the service level, then those for the component level.

Service level keys

On the uppermost level of the service definition, only two keys can be used: name and components.

name

The name of the service as you want to refer to it internally. Naming the service via the definition file is optional. Alternatively it can be named when creating the service, e. g. using swarm create, or an automatically generated name will be assigned.

The service name has to be unique within the environment you’re deploying it to. An attempt to create a service with the same name twice within one environment will result in an error message.

components

This key holds an object containing component definitions. Component definitions are key-value-pairs, where the key is the name of the component and the value holds configuration details as described next under Component Level Keys.

A component usually references a Docker image to be run using the image key. In contrast, there can be components without an image key, which are then pure configuration components. In the complex example above, the components named appserver, datastore/redis and datastore/redisbackup reference images to be run, while the component datastore is a configuration component.

Here is the structure of the complex example again, reduced to only display component names:

{
  "components": {
    "appserver": {…},
    "datastore": {…},
    "datastore/redis": {…},
    "datastore/redisbackup": {…}
  }
}

The example shows how components can have a hierarchical naming structure, pretty much like file system folders and sub-folders. The forward slash / is used as a seperator.

This allows you to create a tree-like structure for the internals of your services. This structure can be used for several purposes:

  • Creating a logical ordering that groups components which belong together under a common parent.
  • Controlling parts of services in a recursive manner, e. g. stopping or starting a certain component with all its descendent components.
  • Forming “Pods”, which allow for the sharing of system resources between components. See the pod key description below.

Component Level Keys

image

The name of a docker image. Images can be referenced from either Giant Swarm’s private registry or the public Docker Registry. Some examples:

  • redis:latest: The latest official Redis image.
  • redis: Same as above. When there is no version tag, :latest is assumed.
  • giantswarm/helloworld: A public GiantSwarm helloworld image
  • registry.giantswarm.io/orgnamespace/webserver:4.2.3: Some image from the private Giant Swarm registry.

In order to address images from the Giant Swarm registry, a fully qualified image name in the form registry.giantswarm.io/<organization_namespace>/<image_name>[:<tag>] is required.

Find out more on working with Giant Swarm’s registry in the registry reference.

entrypoint

The entrypoint key is optional. It can be used to overwrite the default ENTRYPOINT configured in the Docker image. You can provide the path to an executable here, e.g. /opt/app/bin/myapp.

args

The args key is optional and, if present, expects as value an array of strings. These strings will be passed as arguments when your container is started, just like the arguments you can pass to docker run. The effect of these arguments depends on how the container is set up. Please refer to the Dockerfile documentation, especially on the CMD and ENTRYPOINT directives.

This example shows how entrypoint and args can be used together to call an executable with arguments:

"image": "busybox:ubuntu-14.04",
"entrypoint": "/sbin/find",
"args": ["/", "-type", "d"]

This provides for some convenience, since it potentially allows you to use images in a flexible way for more than a single purpose. So it might save you from building and pushing multiple almost identical images.

The links key allows to define network connections from one component to another, which work very much like Docker links. Since links also form dependencies between components, they influence the order in which components are started and stopped.

Example: Component A has a link to component B, thus A depends on B to run. So when the service is started, component B is started first. Component A is started once B is up.

The description in this section explains how links within a service work. To learn more about creating links between services, please refer to the section Linking between services.

Syntax-wise, the value of the links key is an array of objects, each allowing the following keys:

  • component: name of the target component to be linked to. This key is required, a string is expected as value.
  • target_port: TCP port of the target component to connect to. This port has to be defined using the ports key in the target component definition. This key is required, an integer or string is expected as a value.
  • alias: An optional alias name to be used instead of the actual component name for the link. This influences the names of automatically generated environment variables for host and port of the required component. Read on for more information on that.

Let’s review the link defined in our simple example above, which points from the webserver component to the database component:

"links": [
  {
    "component": "database",
    "target_port": 3306
  }
]

The component key here refers to the name of the target component to connect to. The target_port is one of the port defined in the target component.

This link ensures that the database component will be started before the webserver component can start.

In addition, this link definition allows for the creation of a network link (TCP) between the webserver component and port 3306 of the target component database. The connection to the target can be made in different ways:

  • Via the hostname defined by the link’s component or alias value. In the example above, the host name available will be database. This host name will resolve to the target component’s current IP address.
  • Via the IP address obtained via an environment variable. The schema of the name for this variable is <NAME_or_ALIAS>_PORT_<PORT>_ADDR. For the example above, this creates an environment variable DATABASE_PORT_3306_TCP_ADDR. The value of this variable will be the IP address assigned to the target component.

The purpose of the optional alias key in a link definition is to influence the hostname generated for the target component and the environment variable name according to specific requirements. If the alias key is used, it takes precedence over the name and defines how the environment variable and host name for linking will be called.

Suppose we would change the link definition above to this instead:

"links": [
  {
    "component": "database",
    "target_port": 3306,
    "alias": "db"
  }
]

As a result, our component would now provide an environment variable with the name DB_PORT_3306_TCP_ADDR. The according host name would now be db.

A special rule applies when using component names containing the hierarchy seperator /. Let’s consider the following example:

"links": [
  {
    "name": "datastore/redis",
    "port": 6379
  }
]

In this case, only the last part of the component name is used. Our linking component would provide the environment variable REDIS_PORT_6379_TCP_ADDR and the hostname REDIS to connect to. In order to make the naming more explicit, you could of course set an alias as explained above.

domains

The domains key allows you to make components of your service accessible under one or several domain names.

Note: Currently it’s only possible to expose components which support the HTTP protocol.

The example below maps the given component’s port 8080 to the public port 80 using the domain name myexample.gigantic.io:

"domains": {
  "8080": "myexample.gigantic.io"
}

One port can be made available under more than one domain, if needed. Here is an example:

"domains": {
  "8080": [
    "www.example.com",
    "www.example.co.uk",
    "www.example.org"
  ]
}

A component can make more than one port available:

"domains": {
  "8000": "www.example.com",
  "9000": "admin.example.com"
}

The domains key can be used in any number of components.

When deciding on what domain name to use, you have two general options available:

  • Use a subdomain of the gigantic.io domain
  • Use your own domain name

Both options are explained in detail below.

Using the gigantic.io domain

This option is the easiest, especially for services in their early stage.

If you want to make your service available as my-super-app.gigantic.io, fine. Simply enter that string as a key in the domains setting and make it point to the correct port. There is nothing else you have to take care of. (You might want to test before if that subdomain is already in use by someone.)

As of now we do not yet have checks in place to ensure a gigantic.io subdomain is used only by one service. This could theoretically result in another user’s service grabbing a name you want for yourself. This is clearly to be solved and we will come up with a solution sooner rather than later.

Using your own domain name

If you plan to use your own domain name in your configuration, there is one configuration step to take care of on your side: please set up a CNAME entry for the desired subdomain pointing to loadbalancer.giantswarm.io. This can usually be done wherever the DNS settings for that domain are administered.

env

This key allows you to define environment variables which will be available within the running component instances. The key expects an object as value. Values must be of type string. Example:

"env": {
  "STRING_VARIABLE": "foo",
  "NUMERIC_VARIABLE": "123"
}

ports

This key to make TCP ports of a component accessible, as required when a component should be accessible via a link or domain. The value of this key can either be a number or an array.

If, for example, a component should expose TCP port 80, the according value would be:

"ports": 80

Multiple ports can be configured in this way:

"ports": [80, 81]

scale

The scale key can be used to set details on how a component should be scaled to multiple instances. Without specific definition, exactly one instance per component is started during service startup and scaling up is only limited by system-wide limitations.

The Object given with this key can contain the following keys, all of which are optional:

  • min: The minimum number of instances to run for this component
  • max: The maximum number of instances to run for this component
  • placement: Specifying the placement of the instances (see below for details)

The following example will enforce the launch of three instances on startup of the service:

"scale": {"min": 3}

placement

scale can optionally contain a placement key, which specifies the placement of scaled instances of this component on different machines.

The default value for this key is simple. This leaves the placement of instances for a component up to the scheduler, which does not give any guarantees as to placement on different machines. E.g. 3 instances of a scaled component could in cases be scheduled on the same machine.

By setting placement to one-per-machine, you can force the scheduler to schedule each instance of a component on a different machines, which enables actual high availabilty setups.

Note: With one-per-machine no two instances of a component will be started on the same machine. Thus, the upper scaling limit of your component will be automatically set to the number of machines in the cluster. You will get an error if you try scaling above said limit. In case you don’t know the number of machines in your cluster, you can enquire this number with support.

The following example will enforce the launch of three instances on startup of the service, which will all be placed on different machines:

"scale": {"min": 3, "placement": "one-per-machine"}

pod

With the pod key you can define that several components of the same service should be grouped into a pod. This grouping has several effects. All component instances in a pod

  • run on the same machine,
  • share the same IP address,
  • share the same TCP/UDP port space,
  • can communicate via IPC,
  • and can share volumes.

Additionally, if one of the instances in a pod fails, all others instances within this pod will be restarted as well.

Creating a pod

To form a pod, components have to be organized in a hierarchical manner. Then, a component with descendants has to carry the pod key with a string value of either inherit or children.

Our complex example above uses the "pod": "children" definition on the datastore component to group the components datastore/redis and datastore/redisbackup within the same pod. Here is the according excerpt of the definition:

"datastore": {
  "pod": "children",
  ...
},
"datastore/redis": {
  ...
},
"datastore/redisbackup": {
  ...
}

The children setting creates a pod containing only the direct descendants of a component (but not the component itself!). The inherit setting, in contrast, would also include all descendants deeper in the hierarchy into the pod.

Scaling pods

If you scale a component in a pod, the entire pod containing the component will be multiplied. This means that all components belonging to this pod will be scaled to the according number of instances.

Note: The scaling policy (incl. min,max,placement) of each component in a pod needs to be the same for all components within the pod. If you set the scaling policy in one component of a pod it automatically applies to all components within the pod. There’s no need to repeat the scaling policy in each component.

Scaling Pods Illustration

The illustration above shows the effect of scaling a pod. After scaling, two identical pods with distinct identifiers exist. Both contain distinct instances of the same components. Sharing of resources only works within the boundaries of each pod. Instances within one pod do not share any resources with the other pod. Likely, the instances within one pod are not running on the same machine as those of the other pod.

Excluding a component from a pod

It is possible to exclude a component from being a part of a pod by defining the pod key with the string value none on that component. If, for example, a component (a/b/c) is a descendant of a component (a) that defines a pod with the inherit setting, this would allow to prevent (a/b/c) from being part of the pod.

volumes

When a process running in your component instance writes data into the container, this data is not persisted throughout restarts of the instances. So in case of an update, a failure or a manual restart, the data will be lost.

With volumes you can preserve data throughout instance restarts. The volumes you define will be created when the owning instance is created and will be deleted upon instance deletion.

With a normal volume, when scaling a component to more than one instance, each instance will have it’s own distinct volume. This behaviour can be changed by the use of the shared option, described below.

When working with pods, volumes can also be shared between different components. Refer to the pod section further below to learn more.

You can find more details about volumes in the Storage and Volumes reference.

Defining a volume

The volumes key expects an array of simple objects as value, one object for each volume you want to define. Each of these objects must have the following keys:

  • path: The path, in which the volume will be mounted, as a string
  • size: A string defining the volume size in gigabytes in a format like <n> GB.

The complex example above contains volume definitions in several coponents. Here is one excerpt:

"volumes": [
  {
    "path": "/var/data",
    "size": "2 GB"
  }
]

In addition to the mandatory keys, the volume object supports the shared key which expects a boolean value. If the key is ommitted, the value false is assumed. Here is an example:

"volumes": [
  {
    "path": "/var/data",
    "size": "2 GB",
    "shared": true
  }
]

With a shared volume, multiple instances of one component have access to the same file system. Be aware that this can lead to race conditions, where multiple processes try to write to the same file concurrently. As the application developer, you have to take precautions like introducing locking mechanisms to prevent failures.

Reusing all volumes from a component within the same pod

A component can reuse all volumes defined by another component in the same pod by using the volumes-from key together with the component name as a value in the volumes definition.

The example below shows how component main/sub_b reuses all volumes defined in component main/sub_a.

"main": {
  "pod": "children"
},
"main/sub_a": {
  "volumes": [
    {
      "path": "/var/first_dir",
      "size": "2 GB"
    },
    {
      "path": "/var/second_dir",
      "size": "2 GB"
    }
  ],
  ...
}
"main/sub_b": {
  "volumes": [{"volumes-from": "main/sub_a"}],
  ...
}

Reusing a specific volume from a component within the same pod

Instead of reusing all volumes from another component in the pod, components can also reference specific volumes defined by other components. The following keys are supported in this case:

  • volume-from: The name the component in the same pod providing the volume.
  • volume-path: The path of the volume as defined in the component providing the volume.
  • path (optional): If specified, this will be used as mounting point of the volume from the referenced component. If this key is not set, the path as configured in the component defining the volume is used.

The following excerpt of an example definition shows how a volume can be re-used.

"main": {
  "pod": "children"
},
"main/sub_a": {
  "volumes": [
    {
      "path": "/var/first_dir",
      "size": "2 GB"
    },
    {
      "path": "/var/second_dir",
      "size": "2 GB"
    }
  ],
  ...
}
"main/sub_b": {
  "volumes": [
    {
      "volumes-from": "main/sub_a",
      "volume-path"
      "size": "2 GB"
    }
  ],
  ...
}

expose

The expose key allows to publish ports, which are actually offered by descendant components for linking within and between services (more about the latter further below).

Within a service, the [link] concept allows to connect components, but certain restrictions apply. A component (A) can only link to a component (B)

  • if (B) has the same parent component as (A) or
  • when they are both top-level components.

The expose key can only be used on top-level components, i. e. components which don’t have a forward slash (/) in their names.

Our complex example shows this. The appserver component is not allowed to link directly to the datastore/redis component, since they are not both top-level components and they also don’t share a parent component. So to enable the connection, the datastore component has an expose definition that points to the port offered by the datastore/redis component. Here is an excerpt again:

"datastore": {
  "expose": [
    {
      "component": "datastore/redis",
      "target_port": "6379",
      "port": "6379"
    }
  ],
  …
}

Now the appserver component can link to the datastore component’s port 6379, which actually forwards to the datastore/redis component’s port 6379.

The expose key expects as a value an array of objects with the following keys:

  • component: The name of the target component offering the port to connect to
  • target_port: The port offered by the target component
  • port: The port to be exposed in the current component. Each port number can only be used once within the entire service.

Note: If you want to expose ports of a single top-level component, you need change that component to be a descendant to a new top-level component, in which you then use expose like explained above.

Linking between services

Giant Swarm allows to define links between two otherwise independent services running within the same environment, to allow network connections. Links between services differ from links within a service in several ways:

  • Links between services are non-blocking and not validated. If Service (S1) links to a service (S2), the current status of (S2) does not affect the startup of (S1). (S1) will be started even if (S2) is down or doesn’t exist at all. This means that as the developer making use of the connection, you’ll have to implement checks accordingly.
  • Links between services can by cyclic. Service (S1) can point to (S2) and vice versa.

As an example, we show the definition for another service here that has a link to the service defined by our complex example:

{
  "name": "redis_monitor",
  "components": {
    "monitor": {
      "image": "registry.giantswarm.io/myorg/redis-monitor",
      "links": [
        {
          "service": "complex_service",
          "target_port": 6379
        }
      ]
    }
  }
}

Here, the service key has the name of the target service to link to as a value. The target_port refers to the port offered by the expose object in that target service.

Note: If you want to expose ports for linking to another service, expose needs to be used in top-level components that are parent to the components that you want to linked to from the other service.

Manually signal readiness

The component level key signal-ready allows you to change the default assumptions of Giant Swarm’s platform on when a component has started and is considered running. The default setting for this key is false, which results in our platform detecting the start of the process and then conisdering the component running. This is in some cases, e.g. with processes that have a very long initialization time, too early and can result in unexpected behaviour of your service.

By setting signal-ready to true you set a component to manual readiness notification. In this case you have to use the environment variable GIANTSWARM_INSTANCE_API, which contains a URL endpoint as a value, to set the status to ready with a simple HTTP POST request in your component.

In the complex example above, our appserver component has signal-ready set to true. That means that we need to add an HTTP POST request to ${GIANTSWARM_INSTANCE_API}/v1/ready to the code of our component.

Note: Using manual readiness signalling will occupy a single network port inside your container. This port is choosen by the Giant Swarm platform and falls in the range between port 2080 and 3000 (in rare cases the upper range may increase if you have a shared pod with lots of components in them).

Note: Due to a current issue in our infrastructure, containers that are taking a long time starting might cause EOF errors in the CLI or API. This might be more likely to happen with signal-ready set to true. However, the actual actions, e.g. swarm update will still be running through. We are working on fixing this issue soon.

Memory Limit

Each container has a memory limit that is configured per cluster. Our shared cluster for example has a limit of 512 MB. For certain dedicated clusters and on-premise installations, you can also configure a limit yourself, by using the memory-limit field. Please note that there is still an upper and lower boundary, which depends on your cluster installation.

"my-component": {
  "image": "registry.giantswarm.io/myorg/webserver",
  "memory-limit": "2 GB",
  …
}

The format for memory limit is easy: An integer plus a unit. Whitespaces are ignored. Allowed units are KB (1000 Byte), MB (1000 KB), and GB (1000 MB). The units are case-insensitive. Geeky extra: If you are so inclined, you can even use kibi units, e.g. 512 KiB.

Giant Swarm context variables

There are four Giant Swarm context variables available for use in your swarm.json. The respective values of these varialbes are set based on your login and current selected environment. Note that you cannot change the values of these variables unless you change your actual login or environment through CLI or API.

  • GIANTSWARM_ORGANIZATION: Your current selected organization, e.g. “giantswarm”
  • GIANTSWARM_ENVIRONMENT: Your current selected environment, e.g. “dev”
  • GIANTSWARM_ENV: Your current selected organization and environment, e.g. “giantswarm/dev”
  • GIANTSWARM_LOGIN_NAME: Your username that you logged in with into Giant Swarm, e.g. “yourusername”

You can use these anywhere in your swarm.json. E.g. in our complex example above, we use the $GIANTSWARM_LOGIN_NAME variable to customize the domain entries of the service based on the current user’s Giant Swarm username.

Making use of additional configuration variables

Imagine you would like to run an almost identical service in two different environments, say each one with only a different version of an image.

To prevent you from having to copy your entire servcie definition file and make those tiny changes, our service definition supports the use of variables.

The use of variables is explained in more detail in the documentation of the swarm create command.

Further reading