Tag: caching

In the process of performance testing the ability to swap out languages in App Engine detailed in this post, I stumbled on to something.  I was testing performance, and realized that the tests weren’t being accurate because of differences in caching. Ideally, to get the tests to be apples to apples, I would just have to get my PHP code and Go code to use the same Memcache instance and keys.  (I should have written my testing better, but then if I had I would never have stumbled into this.)

To start, follow the steps to get multiple languages working in a production instance or a development instance.

Assuming you are writing from PHP:

$memcache = new Memcache;
$memcache->set("secret", "I wrote this over in PHP");
$value = $memcache->get("secret");
echo $value;

And then to read from Go:

func handler(w http.ResponseWriter, r *http.Request) {
	c := appengine.NewContext(r)

	item, err := memcache.Get(c, "secret")
	if err != nil {
		handleError(w, err)
	}

	fmt.Fprint(w, string(item.Value))
}

It really is that easy. Now the hard part comes when you want to transfer complex data between the two.  Use JSON to encode the objects.  Both languages can handle it pretty effortlessly and Go on App Engine has JSON object handling built in as a codec to its memcache implementation. You could save it in another format like XML then read and write data like a string, while manually encoding and decoding.  You could also staple your had to your desk. Let’s not be a masochist and just do it in JSON – but I suppose it’s your choice.

Once you do that, it’s as simple as encoding to JSON in PHP:

$memcache = new Memcache;

$info['name'] = "hometown";
$info['city'] = "Philadelphia";
$info['state'] = "PA";
$info['latitude'] = "39.995664772727"; 
$info['longitude'] = "-75.138788636364";

$secret = json_encode($info,JSON_NUMERIC_CHECK);

$memcache->set("secret2", $secret);
$value = $memcache->get("secret2");	

echo $value

Then decoding in Go.

type Location struct {
	Name  string  `json:"name"`
	City  string  `json:"city"`
	State string  `json:"state"`
	Lat   float32 `json:"latitude"`
	Lon   float32 `json:"longitude"`
}

func jsonHandler(w http.ResponseWriter, r *http.Request) {
	c := appengine.NewContext(r)

	var item Location
	key := "secret2"
	_, err := memcache.JSON.Get(c, key, &item)
	if err != nil {
		handleError(w, err)
	}

	fmt.Fprintf(w, "%+vn", item)

}

Note a couple of things:

• I omitted graceful memcache miss handling. I did so for brevity. Make sure you wrap your memcache code that handle cache misses.

• If you are not familiar with Go, those ‘json:’ comments aren’t just comments, they’re instructions on how to encode/decode data between Go and JSON.  So you need them, or it won’t work correctly.

• I ran into an issue with the original version of this code because latitude and longitudes were coming out of the database into PHP as strings and not floats. When you went to get them out of memcache in Go, it would through a type mismatch error.  There are 2 solutions to this:

  • cast them correctly to floats before you write to memcache

  • Use JSON_NUMERIC_CHECK in json_encode to get them to write as proper numerics when you write. This seems like the better solution

Why do this?  For starters I was doing it so  both versions of my API could take advantage of caching done by the other language.  But I am sure there are other uses:

• Communication between these modules

• Offloading an expensive data retrieval and processing step to Go then reading memcache from PHP.

• I’m curious if anyone reading has any thoughts.

Note: This will work on either type of memcache solution on App Engine: shared or dedicated. Just make sure you handle cache misses gracefully.

 

All code show here is licensed under Apache 2. For more details find the original source on Github.