In this project, I will be implementing an Image Detection Event Driven architecture based around S3, DynamoDB, Lambda and Amazon Rekognition. Upon uploading an image of a vehicle to S3, the image will be analyzed and the vehicle license plate number will be detected and put into a DynamoDB table. In effect, we are simulating how toll roads would capture a license plate number.
Architecture
Infrastructure
- To quickly create/release the S3 Bucket and DynamoDB table I created this Cloudformation template.
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS CloudFormation S3 Bucket and DynamoDb Table
Parameters:
S3BucketName:
Description: Enter S3 Bucket Name
Type: String
AllowedPattern: "[a-zA-Z0-9]*"
MinLength: '1'
MaxLength: '100'
ConstraintDescription: must contain only alphanumberic characters
DBTableName:
Description: Enter DB Table Name
Type: String
AllowedPattern: "[a-zA-Z0-9]*"
MinLength: '1'
MaxLength: '100'
ConstraintDescription: must contain only alphanumberic characters
HashKeyElementName:
Description: HashType PrimaryKey Name
Type: String
AllowedPattern: "[a-zA-Z0-9]*"
MinLength: '1'
MaxLength: '2048'
ConstraintDescription: must contain only alphanumberic characters
HashKeyElementType:
Description: HashType PrimaryKey Type
Type: String
Default: S
AllowedPattern: "[S|N]"
MinLength: '1'
MaxLength: '1'
ConstraintDescription: must be either S for String or N for Number
RangeKeyName:
Description: HashType Sort Key Name
Type: String
AllowedPattern: "[a-zA-Z0-9]*"
MinLength: '1'
MaxLength: '2048'
ConstraintDescription: must contain only alphanumberic characters
RangeKeyNameType:
Description: HashType Sort Key Type
Type: String
Default: S
AllowedPattern: "[S|N]"
MinLength: '1'
MaxLength: '1'
ConstraintDescription: must be either S for String or N for Number
ReadCapacityUnits:
Description: Provisioned read throughput
Type: Number
Default: '5'
MinValue: '5'
MaxValue: '10000'
ConstraintDescription: must be between 5 and 10000
WriteCapacityUnits:
Description: Provisioned write throughput
Type: Number
Default: '5'
MinValue: '5'
MaxValue: '10000'
ConstraintDescription: must be between 5 and 10000
Resources:
MyS3Bucket:
Type: 'AWS::S3::Bucket'
Properties:
BucketName:
Ref: S3BucketName
myDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName:
Ref: DBTableName
TableClass: STANDARD
AttributeDefinitions:
- AttributeName:
Ref: HashKeyElementName
AttributeType:
Ref: HashKeyElementType
- AttributeName:
Ref: RangeKeyName
AttributeType:
Ref: RangeKeyNameType
KeySchema:
- AttributeName:
Ref: HashKeyElementName
KeyType: HASH
- AttributeName:
Ref: RangeKeyName
KeyType: RANGE
ProvisionedThroughput:
ReadCapacityUnits:
Ref: ReadCapacityUnits
WriteCapacityUnits:
Ref: WriteCapacityUnits
Outputs:
BucketName:
Value:
Ref: MyS3Bucket
Description: Table name of the newly created S3 Bucket
TableName:
Value:
Ref: myDynamoDBTable
Description: Table name of the newly created DynamoDB table
Lambda function
The Lambda Function is created with Python
Triggered via object creation in S3
Utilizes an IAM role with Amazon Rekognition, S3, DynamoDB and Cloudwatch Log permissions.
import json
import boto3
import os
import sys
import uuid
import logging
def lambda_handler(event, context):
logger = logging.getLogger()
logger.setLevel(logging.INFO)
rekognition_client = boto3.client('rekognition')
dynamodb_client = client = boto3.client('dynamodb')
logger.info('Found event{}'.format(event))
for record in event['Records']:
# Read the value of the eventSource attribute.
#
# You can use this to conditionally handle events
# from different triggers in the same lambda function.
event_source = record['eventSource']
logger.info(event_source)
# read S3 bucket and object key
bucket = record['s3']['bucket']['name']
key = record['s3']['object']['key']
min_confidence = 90
logger.info('Found bucket ' + bucket)
logger.info('Found key ' + key)
# use Amazon Rekognition to detect text in the image
rekognition_results = rekognition_client.detect_text(
Image = {'S3Object': {'Bucket': bucket,'Name': key}}, Filters = {'WordFilter': {'MinConfidence': min_confidence}})
# write results of text detection to DynamoDB
for result in rekognition_results['TextDetections']:
vehplate = result['DetectedText']
confidence = str(result['Confidence'])
logger.info('Found text ' + vehplate)
dynamodb_response = dynamodb_client.put_item(
Item={'vehicle': {'S': vehplate},'filename': {'S': key}, 'confidence': {'N':confidence}},
ReturnConsumedCapacity='TOTAL',
TableName='plateindex')
logger.info('DynamDBResponse ' + format(dynamodb_response))
# return the entities that were detected.
return {
'statusCode': 200,
}
Lambda IAM Role
The Lambda Function will need an IAM role that allows:
Writing logs/creating log streams to CloudWatch
Call Amazon Rekognition Detect Text API
Get objects from S3 bucket
Read/write/update to DynamoDB
Execute queries on DynamoDB
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:PutItem",
"dynamodb:GetItem",
"dynamodb:Query",
"dynamodb:UpdateItem",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:dynamodb:*:xxxxxxxxxxxx:table/*",
"arn:aws:logs:*:xxxxxxxxxxxx:log-group:*:log-stream:*"
]
},
{
"Effect": "Allow",
"Action": "rekognition:DetectText",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"logs:CreateLogStream",
"dynamodb:Query",
"logs:CreateLogGroup"
],
"Resource": [
"arn:aws:logs:*:xxxxxxxxxxxx:log-group:*",
"arn:aws:s3:::*/*",
"arn:aws:dynamodb:*:xxxxxxxxxxxx:table/*/index/*"
]
}
]
}
Testing Lambda Function
The Lambda Function will invoke upon uploading the image to S3
Armed with a test image from Google
Upload an image to S3 to invoke the lambda function:
After uploading is complete
- Check the CloudWatch logs to determine if the text was found:
- Lambda Function detected the license plate text and logged it in the log stream.
Check DynamoDB Entry
The highest entry was for the license plate number; however other results were included such as the state name and month/year of tag expiration.
- Exclude results under 99% confidence:
Conclusion
This project was a great learning experience in CloudFormation, CloudWatch Logs, DynamoDB, Amazon Rekognition and Lambda. I thoroughly enjoyed working on this project and look to evolve this architecture in the future.
To allow users to upload to the S3 bucket, create pre-signed URLs to give users access to the S3 bucket.