Two Languages in App Engine Development

In my last post I outlined getting Go and PHP to act as modules in the same App Engine instance.  However I only really tested it on a “production” App Engine instance, I didn’t test it in development, because I typically use the Google App Engine SDK for each respective language separately.

When I tried the combined dispatch.yaml on the Google App Engine SDK for PHP I got the following error on a Mac running OS 10.10.2 (Yosemite):

OSError: [Errno 2] No such file or directory: '/Applications/Development/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/goroot/pkg/tool’

When I tried the combined dispatch.yaml on the Google App Engine SDK for Go I got the following error:

The development server must be started with the --php_executable_path flag set to the path of the php-cgi binary.

And when  used the -php_executable_path option with any of the copies of PHP on my system – including the ones that are buried in the PHP SDK – I got:

_PHPEnvironmentError: No input file specified.

After struggling a bit with this here is the easiest solution I found:

  • Find the location of goroot in the go_appengine folder

  • Find the location of the SDK for php by running

    which dev_appserver.py | xargs ls -l
  • Create a symbolic link to goroot in go_appengine in the PHP SDK folder that contains dev_appserver.py

After that you can test your dispatch file in development by running:

dev_appserver.py dispatch.yaml api_php/app.yaml  api_go/app.yaml

Where api_php is the folder your PHP module is in, and api_go is… well you know what I’m saying.

Now, I went out of my way there to say this was the easiest way of doing it.  Not that it wasn’t a hack, or that it was a supported way of doing it.  But it does work.

All code show here is licensed under Apache 2. For more details find the original source on Github.

Two Languages in One App Engine App

AppEngine_512pxThe other day I was talking to students at a bootcamp about languages. I made the comment that language performance can vary depending on what a particular language is best at doing. When you run into performance issues it can sometimes be helpful to try rewriting pieces of your app in a particular language for a performance boost.

I thought about how that could be done in App Engine. Let’s say I have a section of an application that I wrote in PHP, but it was getting more load than expected, so I need to boost its performance. I want to try and see if Go could give me the boost I need. How hard is that to do?

Please keep in mind all of the caveats here.  Sometimes you can get a boost, sometimes it’s worth exploring. You know, it was a theoretical conversation. And for the record. This need to drop to another language doesn’t have to be performance related. It could be due to SDK or API restrictions, or developer knowledge, or just plain “I want to use another language to do this.”

In App Engine we do this through the use of modules.  Modules allow us to separate front end and back end code from each other.  But they allow us to break up large applications into manageable chunks.  In this case, we’re going to use them to allow us to break up code into multiple languages.

Let’s assume that you have an application with an app.yaml that looks like this:

application: <appengine project name>
module: default
version: 1
api_version: 1
runtime: php55
threadsafe: yes

handlers:
- url: /place
  script: place.php  

- url: /details
  script: details.php   

- url: /distance
  script: distance.php

Let’s say that you want to swap out the distance method for go. The first thing you need to do is write a dispatch.yaml, which looks like this:

application: <appengine project name>

dispatch:
- url: "*/*"
  module: default

This will redirect all calls to your App Engine app to the Php application above. Which is what has been happening to date. But this is a setup step for later.  You then have to add the dispatch file to your application. In a command prompt, from the folder containing dispatch.yaml, run:

appcfg.py --oauth2  update_dispatch .

Write a replacement for your distance method in Go. Go on, we’ll wait…

Ok, assuming you’ve done that you write out an app.yaml for the Go code you wrote:

application:  <appengine project name>
module: goapi
version: 1
runtime: go
api_version: go1

handlers:
- url: /.*
  script: _go_app

Take note of the module name. It has to be different from the original app’s module, which should be “default.”

Once you have all of that handled you need to tweak your dispatch.yaml to replace calls made to the php version of the distance method to the Go method:

application:  <appengine project name>

dispatch:
- url: "*/distance"
  module: goapi

- url: "*/*"
  module: default

Rerun the dispatch update:

appcfg.py --oauth2  update_dispatch .

And there you go, the original PHP service will answer all other calls, but the Go service will answer calls for /distance.

Running multiple language solutions in the same App Engine instance can solve some problems for you.  It also has a few interesting ramifications.  These include the ability to use the same shared Memcached instance between Go and Php. I’m going to show that off in my next blog post.

All code show here is licensed under Apache 2. For more details find the original source on Github.

PHP on App Engine Does cURL

AppEngine_512pxA nice upgrade came about in the 1.9.18 release of App Engine SDK: PHP on App Engine can now support cURL. There are a few caveats that go with it, but it’s a nice step forward.

There are two implementations: cURL_lite and full fledged cURL.

To Enable cURL_lite

  1. Add the directive google_app_engine.enable_curl_lite = “1” to your php.ini file.

Caveats

  • cURL_lite is only allowed to make calls to HTTP or HTTPS clients
  • cURL_lite didn’t work on my local development server without tweaking runtime to php55, but it works for php in production
  • cURL_lite doesn’t require application to have billing enabled

To Enable cURL

  1. Change your runtime setting in your app.yaml from php to php55.
  1. Add the directive extension = “curl.so” to your php.ini file.

Caveats

  • cURL is only available in App Engine’s PHP 5.5 implementation
  • cURL can only be used by applications that have billing enabled
  • cURL is limited by the restrictions of App Engine’s sockets but include:
    • Limited from targeting Google domains
    • May be reclaimed after 2 minutes of inactivity

Now regardless of the implementation, you still call cURL using the “curl_” commands, just the underlying technology changes.

Supporting Documentation

What To Expect from Cloud Security Scanner

I got to experiment a bit with Google Cloud Security Scanner yesterday, and wanted to share with you my experiences, set expectations and what not.

  1. It’s a Front End test. We spin up a bunch of Chrome instances and have them go at your site as a browser.  We aren’t scanning your code on the server side.  We’re testing as if we are on the outside trying to get in.
  2. It’s App Engine only. You get to it through the Developer Console menu for App Engine. It’s not a general purpose scanner.
  3. Read the documentation.  Everything I was confused by for even a moment was noted there. The thing that confused me most was the fact that I ended up getting 150 or so email from my contact form.  Once I understood what was going on, I was all cool with it, but at first I was wondering what the heck was going on.
  4. It’s going to take a while. It scanned 1607 urls on my site in 1 hour 23 minutes. It’s doing a comprehensive scan, while rendering pages in Chrome and running XSS tests. It also limits its requests per second to not become a nuisance.
  5. There is no charge except… The scan does not have a charge associated with it.  However it is making requests of your site, and those requests count against usage and quota. That being said. For me, it didn’t even cause a dent in my usage and quota and I have them all set pretty low. Obviously your mileage may vary depending on the nature of your site.  But for my relative small traffic WordPress blog, running with default quotas, it didn’t cause a blip.

Read more about it, especially the Getting Started Section. If you have an App Engine site, give it a try, fool around with it, and tell us what you think.