Working with Cloud Vision API from JavaScript

I ran into a case where I wanted to fool around with Cloud Vision API from pure JavaScript. Not node.js, just JavaScript running in a browser. There were no samples, so I figured I’d whip up some. So here is a little primer on how to do this from JavaScript in a browser.

First you have to take care of a few prerequisites:

Once you do this you’re ready to start developing. Make sure you hold on to the API key you created above.

The first thing you need to do is create an upload form.  This is pretty basic in HTML5.

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<title>Cloud Vision Demo</title>
	https://ajax.googleapis.com/ajax/libs/jquery/2.2.2/jquery.min.js
	http://key.js
	http://main.js
</head>
<body>
	<form id="fileform" action="">
		<select name="type" id="type">
			<option value="LANDMARK_DETECTION">LANDMARK_DETECTION</option>
		</select><br />
		<input id="fileInput" type="file" name="fileField"><br /><br />
		<input type="submit" name="submit" value="Submit">
	</form>

	
</body> </html>

Note that I’m using a select box to drive the type of detection I am doing.  There are more choices, but I’m sticking with landmark detection for now.

Next you need to convert the image to Base64 encoding to transmit the image data via a REST API. I looked around for how to do this “properly” and the best I came up with was the “easy way” mentioned in this Stack Overflow post – Get Base64 encode file-data from Input Form.

I use  readAsDataURL(). 

function uploadFiles(event) {
  event.stopPropagation(); // Stop stuff happening
  event.preventDefault(); // Totally stop stuff happening

  //Grab the file and asynchronously convert to base64.
  var file = $('#fileInput')[0].files[0];
  var reader = new FileReader()
  reader.onloadend = processFile
  reader.readAsDataURL(file);
}

function processFile(event) {
  var encodedFile = event.target.result;
  sendFiletoCloudVision(encodedFile)
}

Then I massage the content into the JSON format that the Cloud Vision API expects. Note that I strip out “data:image/jpeg;base64,”. Otherwise Cloud Vision sends you errors. And you don’t want that. 

var type = $("#type").val();

  // Strip out the file prefix when you convert to json.
  var json = '{' +
    ' "requests": [' +
    '	{ ' +
    '	  "image": {' +
    '	    "content":"' + content.replace("data:image/jpeg;base64,", "") + '"' +
    '	  },' +
    '	  "features": [' +
    '	      {' +
    '	      	"type": "' + type + '",' +
    '			"maxResults": 200' +
    '	      }' +
    '	  ]' +
    '	}' +
    ']' +
    '}';

And then I send. With the API key.  That’s it. Nothing to it really.

$.ajax({
    type: 'POST',
    url: "https://vision.googleapis.com/v1/images:annotate?key=" + api_key,
    dataType: 'json',
    data: json,
    //Include headers, otherwise you get an odd 400 error.
    headers: {
      "Content-Type": "application/json",
    },

    success: function(data, textStatus, jqXHR) {
      displayJSON(data);
    },
    error: function(jqXHR, textStatus, errorThrown) {
      console.log('ERRORS: ' + textStatus + ' ' + errorThrown);
    }
  });

If you want to dig deeper into the Cloud Vision API you can

The code for all of this is now shared in the Cloud Vision repo on GitHub.

Working with Cloud Vision API from PHP

I have been very excited by the Cloud Vision API recently put into Beta by Google Cloud Platform. I haven’t had a chance to play with it much, and I wanted to fool around with it from PHP on App Engine (or vanilla PHP for that matter), but there is no documentation for PHP yet.

So here is a little primer on how to do this from PHP on App Engine.

First you have to complete a few prerequisites:

Once you do this you’re ready to start developing. Because I am running PHP on App Engine I want the App Engine SDK for PHP.

I’m going to use the GUI to run this app, but you can use the command line just as easily.

cloud-vision-php-gaelauncher

The first thing I need to do is write a php.ini that properly allows use of cURL and has a good limit on uploaded files.

google_app_engine.enable_curl_lite = 1
upload_max_filesize = 5M

Then I set up a page named creds.php to hold my API key for Cloud Storage and my Cloud Storage Bucket name.

<?php 
//Create Bucket here 
// https://cloud.google.com/storage/docs/getting-started-console#create_a_bucket
$bucket = "YOUR BUCKET HERE";
// Get Service account API hereL 
// https://cloud.google.com/vision/docs/getting-started#setting_up_a_service_account
$api_key = "YOUR API KEY HERE ";

 ?>

Then I create a form page named index.php that creates an App Engine Upload URL for me. (If I wanted to not use App Engine, I could just skip the call to Cloud Storage Tools and post directly to the next file in the example: process.php.)

<?php
include_once("creds.php"); // Get $bucket
use googleappengineapicloud_storageCloudStorageTools;

$options = [ 'gs_bucket_name' => $bucket ];
$upload_url = CloudStorageTools::createUploadUrl('/process.php', $options);

?>

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<title>Cloud Vision API PHP Example</title>
</head>
<body>
	<form action="<?php echo $upload_url ?>" method="post" enctype="multipart/form-data">
	Your Photo: <input type="file" name="photo" size="25" />
	<input type="submit" name="submit" value="Submit" />
</form>
</body>
</html>

Then process.php does the hard work of taking the uploaded file, converting it to base64 and uploading to the Cloud Vision API.

<?php

include_once("creds.php"); // Get $api_key
$cvurl = "https://vision.googleapis.com/v1/images:annotate?key=" . $api_key;
$type = "LANDMARK_DETECTION";

//Did they upload a file...
if($_FILES['photo']['name'])
{
	//if no errors...
	if(!$_FILES['photo']['error'])
	{
		$valid_file = true;
		//can't be larger than ~4 MB
		if($_FILES['photo']['size'] > (4024000)) 
		{
			$valid_file = false;
			die('Your file's size is too large.');
		}

		//if the file has passed the test
		if($valid_file)
		{
			//convert it to base64
			$fname = $_FILES['photo']['tmp_name'];
			$data = file_get_contents($fname);
			$base64 = base64_encode($data);
			//Create this JSON
			$r_json ='{
			  	"requests": [
					{
					  "image": {
					    "content":"' . $base64. '"
					  },
					  "features": [
					      {
					      	"type": "' .$type. '",
							"maxResults": 200
					      }
					  ]
					}
				]
			}';

			$curl = curl_init();
			curl_setopt($curl, CURLOPT_URL, $cvurl);
			curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
			curl_setopt($curl, CURLOPT_HTTPHEADER,
				array("Content-type: application/json"));
			curl_setopt($curl, CURLOPT_POST, true);
			curl_setopt($curl, CURLOPT_POSTFIELDS, $r_json);
			$json_response = curl_exec($curl);
			$status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
			curl_close($curl);

			if ( $status != 200 ) {
			    die("Error: $cvurl failed status $status" );
			}

			echo "<pre>";
			echo $json_response;
			echo "</pre>";
		}
	}
	//if there is an error...
	else
	{
		//set that to be the returned message
		echo "Error";
		 die('Drror:  '.$_FILES['photo']['error']);
	}
}
?>

Finally I have to create an app.yaml to serve up the two pages.

module: default
version: 1
api_version: 1
runtime: php55
threadsafe: yes

handlers:

# Needed for static image files

- url: /
  script: index.php

- url: /process.php
  script: process.php

Use GoogleAppEngineLauncher to start your app.

You should get this.

cloud-vision-php-form

Assuming you upload a picture from the top of the Eiffel Tower looking at the Champs de Mars, you’ll get something like this:

et

{
  "responses": [
    {
      "landmarkAnnotations": [
        {
          "mid": "/m/02j81",
          "description": "Champ de Mars",
          "score": 0.81389683,
          "boundingPoly": {
            "vertices": [
              {
                "x": 202,
                "y": 410
              },
              {
                "x": 1967,
                "y": 410
              },
              {
                "x": 1967,
                "y": 1318
              },
              {
                "x": 202,
                "y": 1318
              }
            ]
          },
          "locations": [
            {
              "latLng": {
                "latitude": 48.858249,
                "longitude": 2.294694185256958
              }
            }
          ]
        },
        {
          "mid": "/m/02j81",
          "description": "Paris",
          "score": 0.5426321,
          "boundingPoly": {
            "vertices": [
              {
                "x": 305,
                "y": 412
              },
              {
                "x": 1737,
                "y": 412
              },
              {
                "x": 1737,
                "y": 895
              },
              {
                "x": 305,
                "y": 895
              }
            ]
          },
          "locations": [
            {
              "latLng": {
                "latitude": 48.858546,
                "longitude": 2.3222419999999997
              }
            }
          ]
        },
        {
          "mid": "/g/1tc__sx0",
          "description": "France Eiffel Hotel",
          "score": 0.36458692,
          "boundingPoly": {
            "vertices": [
              {
                "x": 732,
                "y": 394
              },
              {
                "x": 1260,
                "y": 394
              },
              {
                "x": 1260,
                "y": 691
              },
              {
                "x": 732,
                "y": 691
              }
            ]
          },
          "locations": [
            {
              "latLng": {
                "latitude": 48.858362,
                "longitude": 2.294125
              }
            }
          ]
        }
      ]
    }
  ]
}

There you go, bare bones but simple Cloud Vision example in PHP.

If you want to dig deeper into the Cloud Vision API you can

The code for all of this is available on GitHub