Makefile – Get dynamic values from gcloud

Most of the time when I create something in my environment on Google Cloud Platform, I give it a specific name.  For example, I create servers and call them “ThingIWillReferenceLaterWhenIDeleteYou” or more boringly, “Server1.”

Having set names, as I alluded to, makes it easier to clean up after yourself. But there are some cases when you cannot name things when they are created. So it would be nice to get a list of these names. For example, App Engine flexible environment versions for cleaning up after a test.

You can get a list of them with this command:

gcloud app versions list

Which yields this:

SERVICE  VERSION          TRAFFIC_SPLIT  LAST_DEPLOYED              SERVING_STATUS
default  20170405t215321  0.00           2017-04-05T21:53:39-07:00  STOPPED
default  20170405t222022  0.00           2017-04-05T22:20:40-07:00  STOPPED
default  20170405t223620  0.00           2017-04-05T22:36:38-07:00  STOPPED
default  20170405t230438  0.00           2017-04-05T23:04:59-07:00  STOPPED
default  20170405t235759  0.00           2017-04-05T23:58:27-07:00  STOPPED
default  20170407t102935  0.00           2017-04-07T10:29:55-07:00  STOPPED
default  20170407t110623  1.00           2017-04-07T11:06:45-07:00  STOPPED

Now normally I would have to add extra code to my Makefile to rip out the version names.

But gcloud actually has a robust formatting tool. So instead of running the command above I can run:

gcloud app versions list --format='json'

And get the JSON representation, which looks like this:

[
  {
    "environment": {
      "FLEX": null,
      "MANAGED_VMS": {
        "FLEX": null,
        "MANAGED_VMS": null,
        "STANDARD": {
          "FLEX": null,
          "MANAGED_VMS": null,
          "STANDARD": null,
          "name": "STANDARD",
          "value": 1
        },
        "name": "MANAGED_VMS",
        "value": 2
      },
      "STANDARD": {
        "FLEX": null,
        "MANAGED_VMS": {
          "FLEX": null,
          "MANAGED_VMS": null,
          "STANDARD": null,
          "name": "MANAGED_VMS",
          "value": 2
        },
        "STANDARD": null,
        "name": "STANDARD",
        "value": 1
      },
      "name": "FLEX",
      "value": 3
    },
    "id": "20170405t215321",
    "last_deployed_time": {
      "datetime": "2017-04-05 21:53:39-07:00",
      "day": 5,
      "hour": 21,
      "microsecond": 0,
      "minute": 53,
      "month": 4,
      "second": 39,
      "year": 2017
    },
    "project": "redacted",
    "service": "default",
    "traffic_split": 0.0,
    "version": {
      "automaticScaling": {
        "coolDownPeriod": "120s",
        "cpuUtilization": {
          "targetUtilization": 0.5
        },
        "maxTotalInstances": 20,
        "minTotalInstances": 2
      },
      "betaSettings": {
        "cloud_sql_instances": "redacted:us-central1:redacted",
        "has_docker_image": "true",
        "module_yaml_path": "app.yaml",
        "no_appserver_affinity": "true",
        "use_deployment_manager": "true"
      },
      "createTime": "2017-04-06T04:53:39Z",
      "createdBy": "tpryan@google.com",
      "env": "flexible",
      "id": "20170405t215321",
      "name": "apps/redacted/services/default/versions/20170405t215321",
      "runtime": "php",
      "servingStatus": "STOPPED",
      "threadsafe": true,
      "versionUrl": "https://20170405t215321-dot-redacted.appspot.com"
    }
  },
...

Using a JSON parser might make this easier, but there is an even easier way:

cloud app versions list --format='value[terminator=" "](version.id)'

Which yields:

20170405t215321 20170405t222022 20170405t223620 20170405t230438 20170405t235759 20170407t102935 20170407t110623

What will this do?

It will list just the value of version.id and it will separate each record it returns with a ” “, not a line break.  This allows me to drop this generated list into any command that takes multiple names and run them. The gcloud CLI takes multiple arguments in this way. 

So to make this applicable to Makefiles I have to do one more thing – take this data and put it in a variable.

VERSIONLIST = $(shell gcloud app versions list --format='value[terminator=" "](version.id)')

Here we are, ready to use this variable in other Make commands. This works for most of the other places in GCP where you see random values spitting out, like IP forwarding rules, and GKE nodes, to name two. 

To learn more about how to filter and format your gcloud commands, check out the Google Cloud Platform Blog.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s