Building Large Scale,
Microservice-driven Applications
Andrei Papancea ’15
Columbia University, MS Computer Science
NLX Inc, CEO & Co-Founder
Dealing with Large Scale Applications
Platform(s)
Security
Requirements
Cost
Channel(s)
Availability
Goal:
Learn how to build highly available, distributed, and scalable systems that are also cost-effective, using Microservices.
Problems with Monolithic Systems
Microservices: A Solution
Microservice-driven APIs
Scaling the Frontend
Asynchronous Workflows
Scenario:
You have a great idea for a new AI-powered concierge service.
You set up a quick MVP to get your product out there.
Your new website is up!
Frontend
Backend
Logs
Server
Now you scale from 100 users to 100k.
Solution? Scale out.
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Load Balancer
But, adding more servers can get very expensive.
A security backdoor has been discovered in the server software that you use.
Solution? Patch all your instances.
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Ugh, managing servers is time consuming.
You realize your NLP operations consume too much memory.
Your solution? Increase instance memory.
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Frontend
Backend
Logs
Everything starts getting more and more expensive.
Alex, one of the developers, decides to deploy the feature he has been working on all week.
BUT…
…Sam committed non-working code to the master branch.
The new feature can’t be deployed until Sam fixes the bugs. Frustrating.
Challenges with Monolithic Systems
Code complexity and maintainability
Deployment becomes the bottleneck
Fear to change
Lack of ownership
Failure dependencies
One size doesn’t fit all
Hard to scale out
Problems with Monolithic Systems
Microservices: A Solution
Microservice-driven APIs
Scaling the Frontend
Asynchronous Workflows
Microservices to the rescue!
An architectural pattern
Split the application into multiple services that:
Are small
Use simple protocols
Are loosely-coupled
Can be independently deployed
+ each can be written in a different language
Benefits of Microservices
Speed
Faster development & deployments
Innovation
Autonomy of teams, culture of change
Ownership and DevOps culture
Quality
Composability and reusability
More maintainable code
Better scaling and optimizations
Failure Isolation and Resiliency
Microservices++: Serverless Components
No servers to manage
Scalability out of the box
Minimize codebase size
Pay per usage
Extremely low cost (usually fractions of a cent)
Your new best friends.
Lambda
Your new best friends.
Lambda
API Gateway
Your new best friends.
Lambda
API Gateway
Cognito
Your new best friends.
Lambda
API Gateway
Cognito
IAM
Recap:
Microservices are an architectural pattern used to decouple applications
AWS offers lots of different managed services, that can be used as building blocks in your microservice-driven systems
Problems with Monolithic Systems
Microservices: A Solution
Microservice-driven APIs
Scaling the Frontend
Asynchronous Workflows
Let’s build our Concierge Service.
DEMO
Walkthrough of our Concierge app.
Where do I start?
Design. Design. Design.
Not (just) UI design
Stack design
Architecture design
Data structure design
API design
API Design
forces you to think before you build
drives a good chunk of the architecture
drives the data structure design
makes everything more efficient
no more “I’m waiting for the backend to be ready before I can start to implement the frontend”
minimizes time wasted restructuring the API in future versions
Yeah … but building documentation sucks and it is time consuming.
That’s why there’s Swagger.
Swagger
“The world’s most popular API framework”
Powerful tool to design, build, document, and consume REST APIs
Open Source
User friendly
Standardized
Check out http://swagger.io
Swagger + API Gateway =
Seamless API setup
Import the Swagger configuration into API Gateway
endpoints
security settings
request/response models
request/response mapping
response codes, etc.
Swagger + API Gateway + Lambda =
Custom Swagger definitions for Lambda
Set up a fully integrated and managed API
Built-in API management features
API keys
Throttling
Security
Staging, etc.
DEMO
Setup the API using Swagger, API Gateway, Lambda
Showcase Swagger
Go to API Gateway
Create a new API by importing the Swagger config file
Hookup the Lambda functions
Test the function in the endpoint in the Console
Code Deployment with Lambda
Using the AWS Web Console
Using the AWS CLI (preferred)
Bash script
Check out the sample deployment script
https://github.com/mangatanyc/columbia-lecture-concierge/blob/master/backend/deploy.sh
Our Stack so far
Lambda
Lambda
Lambda
Lambda
API Gateway
What’s missing?
Our API is accessible by anyone on the internet.
IAM: Identity and Access Management
Fine-grained access control to AWS resources
including API Gateway
Create roles and permissions
Integrate with your corporate directory
Uses Access and Secret key pairs for access control
Can be used to sign API calls to AWS
Awesome! When was the last time you signed your API requests?
Cognito: User Pools & Identity Federation
Cognito: User Pools & Identity Federation
API Gateway Bonus: SDK Generation
Takes a second to generate
Support for multiple languages
Swift
Obj C
Java
Javascript, and more.
Abstracts all the API calling complexity, including session signing
Cognito + IAM + API Gateway = Security [√]
Cognito exchanges your session for temporary IAM credentials with limited permissions
The API Gateway generated SDK signs API requests using the SigV4 signing process
Verify the identity of the requester
Protect data in transit
Protect against potential replay attacks
Requests are executed with the caller’s credentials
DEMO
Integrate Cognito into the frontend application.
Enable IAM in API Gateway
Show that the API is no longer accessible
Go to the FB post-login callback
Follow the instructions at the URL:
http://docs.aws.amazon.com/cognito/latest/developerguide/facebook.html
Now that’s looking a lot more secure!
Lambda
Lambda
Lambda
Lambda
API Gateway
Cognito
Recap:
Use Swagger to design your APIs and documentation
API Gateway
great API management tool
seamless integration with Swagger
generates SDKs for your API
Lambda
serverless compute service
integrates with API Gateway
Cognito
useful for login workflows
outputs temporary IAM credentials with custom permissions
Great, we have an API. What about the frontend?
Problems with Monolithic Systems
Microservices: A Solution
Microservice-driven APIs
Scaling the Frontend
Asynchronous Workflows
S3: More than just storage
Host your website on S3
HTML, CSS, JS
You get:
99.999999999% of durability
99.99% of availability
You pay:
< $1 per year
DEMO
Host a static website on S3
Upload website files to S3 (manually)
Enable static web hosting
Test URL
Add bucket policy
Test URL again
Upload website files to S3 (with the script)
One more revision of our stack
Lambda
Lambda
Lambda
Lambda
API Gateway
Cognito
S3
Recap:
You can use S3 to host your frontend
S3 hosted websites get out of the box scalability, availability, and durability
OK, let’s make some money.
You want to sell products through your concierge service.
Problems with Monolithic Systems
Microservices: A Solution
Microservice-driven APIs
Scaling the Frontend
Asynchronous Workflows
Current Checkout API
Synchronous
Overloaded
Performs checkout
Sends notification to user
(and in a proper implementation, it would also write to the db)
More prone to failure
Asynchronous Checkout
What?
process credit card transactions asynchronously
Why?
defend against traffic spikes
3rd party services are subject to downtime too
defend against programming errors and bugs
execute intricate order workflows, without impacting the user experience
The Asynchronous Toolset
SQS
The Asynchronous Toolset
Lambda
SQS
The Asynchronous Toolset
SNS
Lambda
SQS
DEMO
Setup the Asynchronous Checkout workflow
Setup the queue
Modify the /checkout code to push the order to the queue
Create a Lambda function from AWS template that polls the queue
Modify the code of the function to send a push notification upon receiving a message from the queue
Place an order as a test
Another Stack Update
Lambda
Lambda
Lambda
Lambda
API Gateway
Cognito
S3
/checkout
SQS
Lambda
SNS
SES
Stripe
Asynchronous Order Workflow .
Push the order object to an SQS queue
Use a Lambda to poll the queue every t minutes
Lambda supports Event triggers, including time based ones
Using Lambda and a 3rd party CC processing service (ex. Stripe), process the CC transaction
If successful, send a Push Notification using SNS and remove the message from SQS (and/or an Email using SES)
Otherwise, try again (the system will do so automatically if you do not remove the message from the queue)
Recap:
Remove complex workflows from your APIs
Leverage SQS, SNS, and Lambda to distribute your application
Queues and notifications make your system a lot more resilient to failure
Microservices: Not a Panacea
Monolithic
Microservices
THANK YOU!
Ideas for next steps...
Integrate a database (ex. DynamoDB) into the system.
the database was omitted from the demo to emphasize how you can build a system service by service, rather than building everything at once
Integrate a distributed cache (Memcached or Redis) to store the NLU state between requests.
Use Route53 to setup the DNS for your domain.
Integrate a real NLP engine and the actual Stripe API to the overall system.
Sources:
https://aws.amazon.com/s3/
https://aws.amazon.com/iam/
https://aws.amazon.com/lambda/
https://aws.amazon.com/cognito/
https://aws.amazon.com/route53/
http://docs.aws.amazon.com/cognito/latest/developerguide/authentication-flow.html
https://aws-de-media.s3.amazonaws.com/images/AWS_Summit_Berlin_2016/sessions/pushing_the_boundaries_1300_microservices_on_aws.pdf
Codebase:
https://github.com/mangatanyc/columbia-lecture-concierge
Steps for building the frontend depicted in the demo:
1. download and setup bootstrap template [12.5 min]
a. http://getbootstrap.com/examples/cover/#
2. create and integrate FB app [15 min]
a. https://developers.facebook.com/docs/facebook-login/web
3. download and setup chat template [30 min]
a. http://codepen.io/supah/pen/jqOBqp
4. create S3 bucket with static website hosting [5 min]
a. http://docs.aws.amazon.com/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html
5. setup AWS profile using CLI [5 min]
a. http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
6. deploy website to S3 [10 min]
a. build a bash script that leverages the AWS CLI to upload your static files to S3
b. https://github.com/mangatanyc/columbia-lecture-concierge/
7. generate Api Gateway SDK [1 min]
8. integrate Api Gateway SDK [15 min]
a. http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-generate-sdk-javascript.html
9. setup Cognito Identity Pool [5 min]
10. integrate Facebook login with the Identity Pool [15 min]
a. http://docs.aws.amazon.com/cognito/latest/developerguide/facebook.html