AWS API Gateway and AWS Lambda Example [Latest]

Table of Contents

AWS API Gateway

1. What is AWS API Gateway?
2. What should one use it for?
3. How does it work – Create and Deploy
4. Advantages
5. Limitations
6. Integrated AWS Services

AWS Lambda

1. What is a Lambda function?
2. Integrate your first Java based Lambda Function with AWS API Gateway

What is AWS API Gateway?

The AWS API Gateway is a relatively new service provided by Amazon to enable users to ‘create’, ‘publish’, ‘monitor’, ‘maintain’ and ‘secure’ their APIs.

It acts as a ‘gateway’ for the end users to access your applications/business logic. If you have an existing public API or are planning to make your application public, you may like to deploy it on the AWS API Gateway to achieve better performance, scalability, availability with low maintenance and cost.

If you are planning to build your business logic/write the application code, you may also like to use AWS Lambda Functions that is another service by Amazon, in addition to deploying it on the API Gateway and get rid of the maintenance of servers.

You can also expose your existing Lambda functions as APIs using the API gateway. Deploying an API doesn’t cost anything. You need to pay on the basis of the number of requests your API receives and the amount of the data it sends back.

What should one use it for?

• To deploy an already public API to achieve better performance, monitoring and throttling.
• To deploy an existing API hosted in AWS (EC2 instances)
• To re-package legacy applications and transform their request/response structures without having to re-write them
• To expose Lambda functions as APIs, avoid managing servers

How does it work – Create and Deploy

You can deploy your existing API on AWS Gateway, if it is deployed/running on Amazon Cloud (on EC2 Instances) or is publicly accessible, no matter where it is deployed.

When you deploy an API, you need to create ‘Resources’ and ‘Methods’ inside those resources. A method in AWS API Gateway corresponds to the HTTP methods such as GET, POST, and DELETE etc. Resources are logical groupings of methods and are like URLs in your web application.

In the API Gateway, a method is a place where you as a developer, would define the structure of your API. Here, you would define the back end that this method will call (a Lambda function or your existing public application) and the request-response structures. The method contains four sections defined in the next section to build this structure.

Below, I’ve noted down the steps you need to take to create and deploy a basic API in AWS API Gateway with simplified information about what each step does.

1. Create an API: This is the first step, once you launch the API Gateway Console. It creates a ‘gate’ for your existing API, or creates a new API on top of your business logic/back-end code or clone an API existing in AWS API gateway.
2. Create a Resource: A resource in the AWS API is similar to a resource in a web application – just as in a web app, you define a URL to access a physical resource, similarly in AWS API, while creating a ‘Resource’, define a URL to access a piece of our physical resources – the back-end code, lambda functions or an existing API. In this step, you just create a logical name, the actual target of this URL is defined while creating a method in the next step.
3. Create a Method: A method corresponds to an HTTP verb like GET, POST, DELETE “inside a Resource”. In a method, you would define your request/response structure for the method call and the end target which will be called when this method is invoked. The end target can be a Lambda function or an existing public application (either deployed on Amazon Cloud or is publicly accessible otherwise).

Now comes the main and tricky part – transformation – which enables users to re-package their existing APIs by changing their request/response structures without re-writing the code. Inside a method, you can define the transformations you want to apply on the request and the response of this method.

In the transformation process, you can change the format of the input and the output and/or add/remove header information.

By using these transformations, you can change the way your existing API interacts with the end users without re-writing it.

A method contains four sections defined below to build its request/response structure:

1. Method Request – Defines who is authorized to access this method and the request structure (headers, body data format through ‘Models’) expected by the AWS API Gateway.
2. Integration Request – Defines the ‘end point’, and the request structure as expected by the end target (the original application). Here, you can have transformations to change your original input request data to a format expected by the end target.
3. Integration Response – Defines the actual response returned by the end target, apply transformations here to change the response body/headers.
4. Method Response – Defines the response returned by the AWS API to the end user (after being transformed if required), you can have different transformations for different response codes.

The first two sections deal with Requests as their names suggest. The ‘Method Request’ section defines the ‘structure of your request as is needed by the API being deployed on the Gateway’. The ‘Integration Request’ section contains the ‘transformations’ you would like to apply on the original request to convert it into the one expected by your back end application.

After the transformations, the Gateway calls your back end application whose URL you defined when you created the method.

The next section, ‘Integration Response’ is the one where you get the response sent by your application. Here, you can apply transformations on this response to make it one which you want to send back to the user. The last section ‘Method Response’ has the transformed response. Here, you can also define custom headers if you want to add some more information in the original response.

Request/Response Transformation
While learning to work with AWS API Gateway, Transformation seemed the trickiest part, not because of the complexity (it’s not very complex), but because of the lack of clear documentation.

Throughout the documentation, Amazon claims that you can convert data from JSON to XML and vice versa, and in each step of the transformation process, they say – ‘specify the data format’ which gives the impression that any valid data type can be transformed, which is not true. I found it works only one way – JSON to XML.

There are three important aspects of the input data and its transformation:

• The data itself – the input data in the JSON format
• The schema or the Model that the input data should follow
• The Mapping template or the transformation using which you can convert the JSON data to a format that our java program expects.

The ‘Model’ is a schema that defines the ‘format’ of the JSON data (like an XSD for an XML data) and the ‘Mapping Template’ defines the transformation to be applied on the JSON d
ata (like XSLT transformations for XML).

Using the Mapping Templates, you can convert the JSON data to any other format. I explain here with a basic example where I had to convert a JSON input to XML for calling a temperature conversion web service:

Define the Model in ‘Method Request’

In the model, define how your input data would look like:

Define the ‘Mapping Template’ in ‘Integration Request’: Here, you define the template that will apply on the input data conforming to the data model defined in the first step and will convert it into an XML which is needed by your end target.

The variable $inputRoot defined in the first line represents the root of the JSON data which you can use to traverse inside the data.

The Result: This set of model and the mapping template will convert the temperature data in JSON format to the XML expected by the end target. And a JSON input like:

Will be converted into an XML input like this:

Apart from converting the data to XML, you can convert it to any other format you want. E.g. in the section – Integrating your Lambda function with the API Gateway, I’ve taken an example to convert a JSON data to a simple integer value which is expected by the Lambda function written in Java.

Deploy the API: After creating the API, its resources and methods, its available for testing through the AWS console. To expose it to your end users, deploy the API to any stage like ‘Testing’ or ‘Production’ or any other stage that makes sense for your project and you’ll get a public URL which you can provide to your end users to call.

While deploying an API, you can enable the cache to further improve performance, enable CloudWatch logs and metrics to monitor the API performance, the ‘Throttling’ Settings to control the traffic on your API before it becomes the bottleneck for your back-end and any client certificate if the Amazon API needs one to connect to your end target.

Advantages

• Deploy APIs without having to manage servers
• Monitor and ‘Throttle’ requests and protect your back-end
• Achieve better performance levels by utilizing AWS Cache
• Achieve reduced latency and protection against Distributed Denial of Server (DDoS) through Amazon CloudFront, ensure high scalability and availability
• Re-package existing API (change its request/response structure and customize headers) without re-writing, though with limited capabilities
• Authorize developers to handle APIs through API keys
• Manage multiple versions of the same API
• Deploy into multiple stages
• Utilize AWS lambda – if you don’t want to have dedicated servers (EC2 Instances) to keep your business logic
• Have custom domain names for your APIs
• No cost of deploying the API, pay only for the API calls and the amount of data it sends out.
• Generate SDKs for Android, iOS and JavaScript
• Low learning curve, integration with other Cloud Services has added advantage for existing users of those services

Limitations

While building a few APIs for my POC, I encountered a big restriction on transformation capabilities. Though the documents at Amazon claim that the requests and responses can be converted from JSON to XML and vice versa, but I found the conversion to be working only one way. The possible conversion is:

• Receive JSON input and convert to XML to send it to your XML-based back-end
• Receive JSON from your back-end (lambda function or any public API) and convert to XML to return to the end user

I really missed the ability to convert an XML response to a JSON format. Why I think it is required is, because if my existing legacy system takes XML as an input, it most likely will return XML as an output. So, if I want to re-package my legacy application to provide support for JSON (a feature highlighted by the documentation), I can only make it work one way – I can take a JSON as input and convert it to XML for my legacy application (which would sit at the back-end of the API Gateway) but I cannot transform the XML returned by my legacy application to a JSON response, because there seems to be no way to convert an XML response to JSON, if I didn’t miss any important detail.

Integrated AWS Services

• AWS Identity and Access Management – to authorize access to APIs
• Amazon CloudFront to manage API traffic
• Amazon CloudWatch – for analysis – have a look at the out of the box metrics provided by CloudWatch on a console
• AWS Lambda – create your back-ends without managing servers
• Amazon Cognito

What is AWS Lambda Function?

Lambda is a compute service provided by Amazon that lets you upload your code/business logic to the AWS infrastructure and manages it. The code that you upload is called the ‘Lambda Function’. Once uploaded, you don’t need to manage the servers required to run your code. You can run this code –

• As an event driven service which triggers on some change like a change in the data of an S3 bucket or a Dynamo DB table.
• As the back-end of an AWS API gateway

I have focused on the second use case here.

Integrate you first Java-based AWS Lambda Function with AWS API Gateway

In the AWS document, it’s clear how to create your first Java-based Lambda function. As a separate activity, calling a Lambda function through the AWS API Gateway is also described in easy to follow steps.

But the problem that I faced, is to integrate both these activities, i.e. to write a java based Lambda function and then call it through the API Gateway, the reason being the AWS documentation’s focus on JSON. From the present documentation, it’s easy to deploy and test your API or a Lambda function if it takes JSON as input and gives JSON back, but it is not easy, at least not directly, to create an API in the gateway that has a Lambda function as it’s backend but does not take JSON as its input.

Below I am presenting an example of a JSON data Model and the Mapping Template I used to convert the JSON input to a simple integer value which was the input of my java based Lambda Function.

Mapping Template:

This mapping template converted the JSON data:

to a simple integer value – 1234 which I passed to my java based Lambda function.

That’s all for now, we will look into more AWS features in coming posts.