Atlassian Connect add-on Starter Kit for App Engine

atlassian_gcpAtlassian Connect add-ons are extensions written by third-party developers to augment Atlassian’s hosted software. We’d love for the Atlassian Community to host their add-ons with Google Cloud Platform.  To help out, I’m releasing Atlassian Connect Add On Starter Kit for App Engine on github.

Google App Engine is a great environment to get started hosting this sort of solution.

  • You just upload code.
  • We handle scaling for you.
  • App Engine serves up webservices with little to no config.
  • App Engine integrates well with other Google Developer Services

On the Atlassian side, as long as you write your services to respond the way Connect expects,  you can write your add-ons in whatever language or technology stack you want.

I wrote the Atlassian Connect Add On Starter Kit for App Engine to give a working example for getting started on App Engine using each of the supported languages.  Each language folder (Go, Java, Php, Python) contains code to create two applications:

Hello World – basically the Hello World example written by Atlassian with added App Engine configs to show the delta between a vanilla add-on and one configured to run on App Engine.

The Hello World. Nothing special.

Language Check – this application scans a JIRA instance for issue titles in languages that differ from the default language, and then labels which language they are.  The idea is that you can then easily triage issues that should be routed to staff who can support that language. It takes advantage of the Google Translate API to accomplish this.

Add-on translating with the default language set to English.
Add-on translating with the default language set to Czech.

Now, all of this isn’t to say you cannot use other technologies in Google Cloud Platform to host your add-on.  You can also host on either Compute Engine or Container Engine. After you configure one of these solutions you can just follow Atlassian’s existing tutorials to create an add-on.

Anyway you want to do it, Google Cloud Platform can help the Atlassian developer community to host their applications.  If you want to write some code and have server configuration and scaling handled by us, App Engine is a good choice.  If you would rather have more control over or flexibility with the systems you run, either Compute Engine or Container Engine can be your answer.

Atlassian Connect Add On Starter Kit for App Engine

Google Cloud Platform at Google I/O

Cloudplatform-at-IOGreat news everyone: we’re going on the road! Google Cloud Platform will be running a series of events this summer called Google Cloud Platform: Next to bring a world tour level of attention to everything we are doing in the Cloud.

This week is Google I/O. The Cloud Platform team will be at Google I/O in a few sessions and events, but you may find fewer of us on the floor and speaking at Moscone Center. However my co-workers, the Developer Advocacy team, will be overflowing to space at Galvanize SF, which is just a few blocks away. The rest of our extended team is hard at work preparing for Google Cloud Platform: Next.  We can’t wait to see you all on tour.

In the meantime, you will be able to find members of the Cloud Platform team at these sessions at Google I/O:


1:00PM – 2:00PM

Room 1 (L2)

Google Cloud Messaging 3.0


1:00PM – 1:30PM

Alcove 3 (L2)

Containers to back your mobile app


1:30PM – 2:00PM

Develop Talk 3 (L2)

Mobile app quality leaps to the cloud


3:00PM – 3:30PM

Develop Talk 3 (L2)

Building a real-time app in 5 minutes with Firebase


9:00AM – 9:30AM

Earn & Engage Talk (L2)

Real-time analytics for mobile and IoT


11:30AM – 12:00PM

Alcove 3 (L2)

Containers to back your mobile app


2:00PM – 2:30PM

Earn & Engage Talk (L2)

Building a real-time app in 5 minutes with Firebase


2:30PM – 3:00PM

Design Talk (L2)

Real-time analytics for mobile and IoT


3:30PM – 4:00PM

Develop Talk 3 (L2)

Mobile app quality leaps to the cloud

As mentioned, we will also be holding an event at Galvanize, from 1-9PM on day 1 (Thursday) of Google I/O.  Members of the Cloud Platform Developer Advocate team will be there for the entire event, and speaking in the evening.

1:00PM – 5:00PM

Tech Stop
Consult with Google Cloud Platform Developer Advocates

5:30PM – 6:00PM

Choosing between App Engine, Compute and Managed VMs
Understand the differences between Google Cloud Platform’s computing options, to make the best choice for your app.
Terry Ryan, Developer Advocate, Google

6:00PM – 6:30PM

Making the real world talk to us in real-time with mobile video
Carter Maslan, CEO, Camio

6:30PM – 7:00PM

Google for Entrepreneurs
Mary Grove, Director of Google for Entrepreneurs

7:00PM – 9:00PM

Drinks, food, chat

Finally, we’ll be hanging out at the various satellite events and in hotel bars in the evening. Track us down on Twitter – you can find us using @googlecloud’s list on Twitter, and when you find one of us, bring your questions, share your ideas, and pester us for swag! Enjoy I/O and we can’t wait to see you at Google Cloud Platform: Next.

Managed VM Not Connecting to on OS X

Ran into this issue today fooling around with Docker and Managed VMs on Google Cloud Platform. I was running on Mac OS X, which meant boot2docker was also in the mix. Figured this could help someone else because it baffled me for a bit.

I was trying to start up a standard runtime for python on Managed VM. I was using the following app.yaml:

module: default
runtime: python27
vm: true

api_version: 1
threadsafe: yes

  cpu: .5
  memory_gb: 1.3

  instances: 1

- url: .*

I ran the gcloud command:

gcloud preview app run ./app.yaml

In the midst of the output, this error was buried:

ERROR    2015-05-22 17:21:59,043] v1 ping attempt failed
with error: Get dial tcp: i/o timeout. If this
private registry supports only HTTP or HTTPS with an unknown CA certificate,
please add `--insecure-registry` to the daemon's arguments. In the
case of HTTPS, if you have access to the registry's CA certificate, no need
for the flag; simply place the CA certificate at

What was really frustrating is that this had worked yesterday and I had changed nothing in between.  I tried a whole bunch of things. I fired a whole lot of searches on Google. In my searches I found this thread on github that suggests it’s an issue with some sort of caching in Docker. So I did the following:

  • Launched Virtual Box
  • Right clicked boot2docker-vm
  • Chose Close ->Power Off
  • Launched boot2docker again via Applications->boot2docker
  • Tried again

And now it works.

Hope this helps someone else.

When to Pick Google Bigtable vs Other Cloud Platform Databases

dbsRecently, Google Cloud Platform announced the availability of an additional database option for our customers: Google Cloud Bigtable.  Now Cloud Platform Developers have another Google managed solution for storing their data. So, when do you use which tool?

The contenders:

Cloud SQL

Are you using some sort of relational database with tables, views, and  indices? Does your application rely on stored procedures and custom table views or write joins? Do you need the certainty of transactions and ACID compliance?  Do you generally both read and write to the data, not in equal amounts, but not lopsidedly one or the other?  If you answered “yes” to a lot of these you probably want to investigate or stick with Cloud SQL, which as its name suggests, is an implementation of MySQL hosted by Google.

Cloud SQL does have all of the limitations of MySQL. Certain types of applications don’t require the complexity of normalized data with no duplication. Other cases require scaling in a manner that SQL cannot handle without additional complexity, reduced performance, or higher cost. Going into the full pros and cons of SQL vs NoSQL is beyond the scope of this post, but rest assured there are reasons that both of them exist, and both are valid choices depending on circumstances.

NoSQL: Datastore or Bigtable

If you are leaning towards a NoSQL solution you now have two Google managed NoSQL choices on our platform. What is the differentiator between Cloud Datastore vs Cloud Bigtable? They are both NoSQL solutions. Both are described as massively scalable.  Both leave you with little to no management (or No Ops for those of you playing buzzword bingo).  The answer lies in four areas:

  • Size
  • Structure
  • Analysis
  • Interface


Bigtable is optimized for mind boggling huge sets of data. Seriously, it is most cost effective when dealing with datasets that start at 1 Terabyte. Datastore can handle large data sets too, but Datastore is performance and cost optimized to handle smaller sets of data too.  Have a few GBs of data? Datastore would be the better call. Have data that might start out small, but grow to a Terabyte in time, still Datastore. Have data that starts at a Terabyte and will keep expanding? Then you’ve started down a path that might make Bigtable interesting. But size alone isn’t the only factor.


Bigtable stores data in a big honkin’ table. Yeah, the name is a little on the nose, but it’s true. There are rows and columns somewhat like relational database systems but not exactly. But it has a schema, and predefined structure.

Cloud Datastore, on the other hand, is more optimal for ad hoc storage of structured data representing objects.  Basically you define an object and then push it into Datastore. You don’t define a schema, create tables, or set up any other sort of structure before storing a record.


Do you need to analyze the data in massive aggregate scale while the database is still online and taking requests? Do you want to run MapReduce on your production data without copying it somewhere for study? Do you want to hook it up to various Big Data analysis toolkits?  If this sounds like what you want to do, Bigtable makes more sense.


If you are coming to Google Cloud Platform from other technologies, and are working with HBase, Bigtable is for you.  Bigtable is accessible through extensions to the HBase 1.0 API and is therefore compatible with a lot of the Hadoop ecosystem as well as other Big Data tools.

On the other hand,  there are also a few limitations. You cannot join. There is no SQL interface. The API gives you Put/Get/Delete individual records, or you can run Scan operations.

Datastore does not have SQL either, but has an API called GQL that while not exactly the same does abstract querying objects in a way that most SQL developers should be able to quickly understand.


Finally the product page has a great explanation of Bigtable’s relation to other Google Cloud Platform offerings:

Cloud Bigtable and other storage options

Cloud Bigtable is not a relational database; it does not support SQL queries or joins, nor does it support multi-row transactions. Also, it is not a good solution for small amounts of data (< 1 TB).

  • If you need full SQL support for an online transaction processing (OLTP) system, consider Google Cloud SQL.

  • If you need interactive querying in an online analytical processing (OLAP) system, consider Google BigQuery.

  • If you need to store immutable blobs larger than 10 MB, such as large images or movies, consider Google Cloud Storage.

  • If you need to store highly structured objects, or if you require support for ACID transactions and SQL-like queries, consider Cloud Datastore.

In short, there is a lot of awesome stuff about Cloud Bigtable, but it doesn’t mean that it is right in all cases.  It’s a NoOps, NoSQL, Big Data analysis tool, meant to be used at massive scale in conjunction with other Big Data tools.  I recommend that you check out the documentation for Bigtable as there is much more to be found there. And let me know if you need more clarification on anything.

PHP on App Engine Updating to PHP 5.5

A few weeks ago Google Cloud Platform released an update for PHP on App Engine that enabled PHP 5.5 on App Engine. It was all very exciting, and there was a forum post about it any everything.  At the bottom though there is a little note:

After 16th April, 2015 we will begin automatically migrating all applications to the php55 runtime.

You may have also seen emails from  “Google App Engine” reminding you of this.  In those notices it has been changed to:

in approximately 2 weeks we will begin automatically migrating your application over to use the PHP 5.5

If you need an extension on PHP 5.4 you can fill out a form to request one.

So this change is coming, and this post is yet another heads up to remind people.

Lumen on App Engine


Laravel announced today that they are launching a new PHP framework named Lumen for building API’s and microservices.  It has an emphasis on speed and it is compatible with a subset of Laravel, making for easy migrations to the larger framework.

I wanted to see how easy it would be to run a Lumen app on App Engine, and I fooled around with it a bit. You just do a simple setting in app.yaml to route all of the apps traffic to the default handler for the Lumen app:

application: [your application id]
version: one
runtime: php55
api_version: 1


- url: /.*
  script: /[path_to_lumen_folder]/public/index.php

With that I got a basic instance running. I did run into one problem though.  The logger tried to write to disk, which is a no-no for App Engine. If this happens you will get an error like this:

Fatal error: Uncaught exception 'UnexpectedValueException' with message 'The stream or file "[Path to your lumen app]/storage/logs/lumen.log" could not be opened

One little configuration tweak got it working though. In [path to your lumen app ]vendor/laravel/lumen-framework/src/Application.php, you have to tweak the logger a bit.

First you add a reference to the right library:

use MonologHandlerSyslogHandler;

And then replace the function getMonologHandler() with this:

protected function getMonologHandler()
return new SyslogHandler('intranet', 'user', Logger::DEBUG, false, LOG_PID);

This advice was taken from the guide for Laravel on PHP for App Engine.

All in all, it is a relatively quick little framework, and having using App Engine to scale out the services it provides seems like a no-brainer.

All code shown here is licensed under Apache 2. For more details find the original source on Github.

Sharing Memcache Between Languages in App Engine

memcache diagramIn the process of performance testing the ability to swap out languages in App Engine detailed in this post, I stumbled on to something.  I was testing performance, and realized that the tests weren’t being accurate because of differences in caching. Ideally, to get the tests to be apples to apples, I would just have to get my PHP code and Go code to use the same Memcache instance and keys.  (I should have written my testing better, but then if I had I would never have stumbled into this.)

To start, follow the steps to get multiple languages working in a production instance or a development instance.

Assuming you are writing from PHP:

$memcache = new Memcache;
$memcache->set("secret", "I wrote this over in PHP");
$value = $memcache->get("secret");
echo $value;

And then to read from Go:

func handler(w http.ResponseWriter, r *http.Request) {
	c := appengine.NewContext(r)

	item, err := memcache.Get(c, "secret")
	if err != nil {
		handleError(w, err)

	fmt.Fprint(w, string(item.Value))

It really is that easy. Now the hard part comes when you want to transfer complex data between the two.  Use JSON to encode the objects.  Both languages can handle it pretty effortlessly and Go on App Engine has JSON object handling built in as a codec to its memcache implementation. You could save it in another format like XML then read and write data like a string, while manually encoding and decoding.  You could also staple your had to your desk. Let’s not be a masochist and just do it in JSON – but I suppose it’s your choice.

Once you do that, it’s as simple as encoding to JSON in PHP:

$memcache = new Memcache;

$info['name'] = "hometown";
$info['city'] = "Philadelphia";
$info['state'] = "PA";
$info['latitude'] = "39.995664772727"; 
$info['longitude'] = "-75.138788636364";

$secret = json_encode($info,JSON_NUMERIC_CHECK);

$memcache->set("secret2", $secret);
$value = $memcache->get("secret2");	

echo $value

Then decoding in Go.

type Location struct {
	Name  string  `json:"name"`
	City  string  `json:"city"`
	State string  `json:"state"`
	Lat   float32 `json:"latitude"`
	Lon   float32 `json:"longitude"`

func jsonHandler(w http.ResponseWriter, r *http.Request) {
	c := appengine.NewContext(r)

	var item Location
	key := "secret2"
	_, err := memcache.JSON.Get(c, key, &item)
	if err != nil {
		handleError(w, err)

	fmt.Fprintf(w, "%+vn", item)


Note a couple of things:

  • I omitted graceful memcache miss handling. I did so for brevity. Make sure you wrap your memcache code that handle cache misses.
  • If you are not familiar with Go, those ‘json:’ comments aren’t just comments, they’re instructions on how to encode/decode data between Go and JSON.  So you need them, or it won’t work correctly.

  • I ran into an issue with the original version of this code because latitude and longitudes were coming out of the database into PHP as strings and not floats. When you went to get them out of memcache in Go, it would through a type mismatch error.  There are 2 solutions to this:

    • cast them correctly to floats before you write to memcache

    • Use JSON_NUMERIC_CHECK in json_encode to get them to write as proper numerics when you write. This seems like the better solution

Why do this?  For starters I was doing it so  both versions of my API could take advantage of caching done by the other language.  But I am sure there are other uses:

  • Communication between these modules

  • Offloading an expensive data retrieval and processing step to Go then reading memcache from PHP.

  • I’m curious if anyone reading has any thoughts.

Note: This will work on either type of memcache solution on App Engine: shared or dedicated. Just make sure you handle cache misses gracefully.


All code show here is licensed under Apache 2. For more details find the original source on Github.


Two Languages in App Engine Development

In my last post I outlined getting Go and PHP to act as modules in the same App Engine instance.  However I only really tested it on a “production” App Engine instance, I didn’t test it in development, because I typically use the Google App Engine SDK for each respective language separately.

When I tried the combined dispatch.yaml on the Google App Engine SDK for PHP I got the following error on a Mac running OS 10.10.2 (Yosemite):

OSError: [Errno 2] No such file or directory: '/Applications/Development/’

When I tried the combined dispatch.yaml on the Google App Engine SDK for Go I got the following error:

The development server must be started with the --php_executable_path flag set to the path of the php-cgi binary.

And when  used the -php_executable_path option with any of the copies of PHP on my system – including the ones that are buried in the PHP SDK – I got:

_PHPEnvironmentError: No input file specified.

After struggling a bit with this here is the easiest solution I found:

  • Find the location of goroot in the go_appengine folder

  • Find the location of the SDK for php by running

    which | xargs ls -l
  • Create a symbolic link to goroot in go_appengine in the PHP SDK folder that contains

After that you can test your dispatch file in development by running: dispatch.yaml api_php/app.yaml  api_go/app.yaml

Where api_php is the folder your PHP module is in, and api_go is… well you know what I’m saying.

Now, I went out of my way there to say this was the easiest way of doing it.  Not that it wasn’t a hack, or that it was a supported way of doing it.  But it does work.

All code show here is licensed under Apache 2. For more details find the original source on Github.

Two Languages in One App Engine App

AppEngine_512pxThe other day I was talking to students at a bootcamp about languages. I made the comment that language performance can vary depending on what a particular language is best at doing. When you run into performance issues it can sometimes be helpful to try rewriting pieces of your app in a particular language for a performance boost.

I thought about how that could be done in App Engine. Let’s say I have a section of an application that I wrote in PHP, but it was getting more load than expected, so I need to boost its performance. I want to try and see if Go could give me the boost I need. How hard is that to do?

Please keep in mind all of the caveats here.  Sometimes you can get a boost, sometimes it’s worth exploring. You know, it was a theoretical conversation. And for the record. This need to drop to another language doesn’t have to be performance related. It could be due to SDK or API restrictions, or developer knowledge, or just plain “I want to use another language to do this.”

In App Engine we do this through the use of modules.  Modules allow us to separate front end and back end code from each other.  But they allow us to break up large applications into manageable chunks.  In this case, we’re going to use them to allow us to break up code into multiple languages.

Let’s assume that you have an application with an app.yaml that looks like this:

application: <appengine project name>
module: default
version: 1
api_version: 1
runtime: php55
threadsafe: yes

- url: /place
  script: place.php  

- url: /details
  script: details.php   

- url: /distance
  script: distance.php

Let’s say that you want to swap out the distance method for go. The first thing you need to do is write a dispatch.yaml, which looks like this:

application: <appengine project name>

- url: "*/*"
  module: default

This will redirect all calls to your App Engine app to the Php application above. Which is what has been happening to date. But this is a setup step for later.  You then have to add the dispatch file to your application. In a command prompt, from the folder containing dispatch.yaml, run: --oauth2  update_dispatch .

Write a replacement for your distance method in Go. Go on, we’ll wait…

Ok, assuming you’ve done that you write out an app.yaml for the Go code you wrote:

application:  <appengine project name>
module: goapi
version: 1
runtime: go
api_version: go1

- url: /.*
  script: _go_app

Take note of the module name. It has to be different from the original app’s module, which should be “default.”

Once you have all of that handled you need to tweak your dispatch.yaml to replace calls made to the php version of the distance method to the Go method:

application:  <appengine project name>

- url: "*/distance"
  module: goapi

- url: "*/*"
  module: default

Rerun the dispatch update: --oauth2  update_dispatch .

And there you go, the original PHP service will answer all other calls, but the Go service will answer calls for /distance.

Running multiple language solutions in the same App Engine instance can solve some problems for you.  It also has a few interesting ramifications.  These include the ability to use the same shared Memcached instance between Go and Php. I’m going to show that off in my next blog post.

All code show here is licensed under Apache 2. For more details find the original source on Github.

PHP on App Engine Does cURL

AppEngine_512pxA nice upgrade came about in the 1.9.18 release of App Engine SDK: PHP on App Engine can now support cURL. There are a few caveats that go with it, but it’s a nice step forward.

There are two implementations: cURL_lite and full fledged cURL.

To Enable cURL_lite

  1. Add the directive google_app_engine.enable_curl_lite = “1” to your php.ini file.


  • cURL_lite is only allowed to make calls to HTTP or HTTPS clients
  • cURL_lite didn’t work on my local development server without tweaking runtime to php55, but it works for php in production
  • cURL_lite doesn’t require application to have billing enabled

To Enable cURL

  1. Change your runtime setting in your app.yaml from php to php55.
  1. Add the directive extension = “” to your php.ini file.


  • cURL is only available in App Engine’s PHP 5.5 implementation
  • cURL can only be used by applications that have billing enabled
  • cURL is limited by the restrictions of App Engine’s sockets but include:
    • Limited from targeting Google domains
    • May be reclaimed after 2 minutes of inactivity

Now regardless of the implementation, you still call cURL using the “curl_” commands, just the underlying technology changes.

Supporting Documentation