Amplify+Angular+Recognitionを使って画像からテキストを読み取るアプリケーションをサクッと作ってみる

どうも!大阪オフィスの西村祐二です。

今回はAngularとAmplifyとRecognitionを使って画像からテキストを読み取るアプリケーションを作ってみたいと思います。

ゴールとなるアプリケーションは下記になります。

文字が含まれる画像をアップロードするとバックエンドのRecognitionのAPIをコールし、画像からテキストを抽出して表示するというアプリケーションです。

作ってみる

環境

  • aws-amplify: 2.2.2
  • amplify cli: 4.12.0
Angular CLI: 8.3.23
Node: 12.13.0
OS: darwin x64
Angular: 8.2.14
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.803.23
@angular-devkit/build-angular     0.803.23
@angular-devkit/build-optimizer   0.803.23
@angular-devkit/build-webpack     0.803.23
@angular-devkit/core              8.3.23
@angular-devkit/schematics        8.3.23
@angular/cli                      8.3.23
@ngtools/webpack                  8.3.23
@schematics/angular               8.3.23
@schematics/update                0.803.23
rxjs                              6.4.0
typescript                        3.5.3
webpack                           4.39.2

セットアップ

Angular CLIを使って、雛形アプリケーションを作成し、Amplify CLIとライブラリをインストールします。

$ ng new text-from-image --style=scss --routing
$ cd text-from-image
$ npm install -g @aws-amplify/cli
$ npm install aws-amplify

amplifyの設定

amplify initコマンドを実行してセットアップします。対話形式で設定していきます。

❯ amplify init
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project text-from-image
? Enter a name for the environment dev
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using angular
? Source Directory Path:  src
? Distribution Directory Path: dist
? Build Command:  npm run-script build
? Start Command: ng serve
Using default provider  awscloudformation

For more information on AWS Profiles, see:
https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html

? Do you want to use an AWS profile? Yes
? Please choose the profile you want to use cm

そのあと、CloudFormationが実行されてRoleやらS3バケットやら必要なリソースが作成されます。下記が出力されたログになります。

Adding backend environment dev to AWS Amplify Console app: xxxxxxx
⠏ Initializing project in the cloud...

CREATE_IN_PROGRESS AuthRole                           AWS::IAM::Role             Mon Jan 20 2020 23:40:03 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS DeploymentBucket                   AWS::S3::Bucket            Mon Jan 20 2020 23:40:02 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS AuthRole                           AWS::IAM::Role             Mon Jan 20 2020 23:40:01 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS UnauthRole                         AWS::IAM::Role             Mon Jan 20 2020 23:40:01 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS amplify-text-from-image-dev-233955 AWS::CloudFormation::Stack Mon Jan 20 2020 23:39:58 GMT+0900 (GMT+09:00) User Initiated
⠙ Initializing project in the cloud...

CREATE_IN_PROGRESS DeploymentBucket AWS::S3::Bucket Mon Jan 20 2020 23:40:05 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS UnauthRole       AWS::IAM::Role  Mon Jan 20 2020 23:40:03 GMT+0900 (GMT+09:00) Resource creation Initiated
⠏ Initializing project in the cloud...

CREATE_COMPLETE UnauthRole AWS::IAM::Role Mon Jan 20 2020 23:40:20 GMT+0900 (GMT+09:00)
CREATE_COMPLETE AuthRole   AWS::IAM::Role Mon Jan 20 2020 23:40:20 GMT+0900 (GMT+09:00)
⠋ Initializing project in the cloud...

CREATE_COMPLETE DeploymentBucket AWS::S3::Bucket Mon Jan 20 2020 23:40:25 GMT+0900 (GMT+09:00)
⠹ Initializing project in the cloud...

CREATE_COMPLETE amplify-text-from-image-dev-233955 AWS::CloudFormation::Stack Mon Jan 20 2020 23:40:28 GMT+0900 (GMT+09:00)
✔ Successfully created initial AWS cloud resources for deployments.
✔ Initialized provider successfully.
Initialized your environment successfully.

Your project has been successfully initialized and connected to the cloud!

Some next steps:
"amplify status" will show you what you've added already and if it's locally configured or deployed
"amplify <category> add" will allow you to add features like user login or a backend API
"amplify push" will build all your local backend resources and provision it in the cloud
“amplify console” to open the Amplify Console and view your project status
"amplify publish" will build all your local backend and frontend resources (if you have hosting category added) and provision it in the cloud

Pro tip:
Try "amplify add api" to create a backend API and then "amplify publish" to deploy everything

Cognitoの設定をします。

設定項目の詳細はこちらをご確認ください。

❯ amplify add auth
Using service: Cognito, provided by: awscloudformation

 The current configured provider is Amazon Cognito.

 Do you want to use the default authentication and security configuration? Default configuration
 Warning: you will not be able to edit these selections.
 How do you want users to be able to sign in? Username
 Do you want to configure advanced settings? No, I am done.
Successfully added resource textfromimaged2d3ad30 locally

Some next steps:
"amplify push" will build all your local backend resources and provision it in the cloud
"amplify publish" will build all your local backend and frontend resources (if you have hosting category added) and provision it in the cloud

Recognitionの設定をします。

設定した項目の詳細はこちらをご確認ください。

❯ amplify add predictions
? Please select from one of the categories below Identify
? What would you like to identify? Identify Text
? Provide a friendly name for your resource identifyText6d7bda0d
? Would you also like to identify documents? Yes
? Who should have access? Auth and Guest users
Successfully updated auth resource locally.
Successfully added resource identifyText6d7bda0d locally

Some next steps:
"amplify push" builds all of your local backend resources and provisions them in the cloud
"amplify publish" builds all of your local backend and front-end resources (if you added hosting category) and provisions them in the cloud

AWSのリソース作成

amplify pushコマンドを実行すると上記で生成されたテンプレートをもとにCloudFormationが実行されてAWSのリソースが作成されます。

もし、Lambdaのランタイムnodejsに関するエラーが出力されてデプロイできない場合はAmplify CLIのバージョンが古い可能性があります。(私はこれでハマりました。)

CLIを最新版にしてい再度試してみてください。

Issueがこちらにあがっていたのでこちらも参考にしてみてください。

The runtime parameter of nodejs8.10 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs12.x) while creating or updating functions. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 97fe6402-c986-4017-8cc6-26e6e12264dc)
⠼ Updating resources in the cloud. This may take a few minutes...

コマンド実行し、成功した際のログを記載しておきます。

❯ amplify push
Scanning for plugins...
Plugin scan successful
✔ Successfully pulled backend environment dev from the cloud.

Current Environment: dev

| Category    | Resource name         | Operation | Provider plugin   |
| ----------- | --------------------- | --------- | ----------------- |
| Auth        | textfromimaged2d3ad30 | Create    | awscloudformation |
| Predictions | identifyText6d7bda0d  | Create    | awscloudformation |
? Are you sure you want to continue? Yes
⠇ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS predictionsidentifyText6d7bda0d    AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:29 GMT+0900 (GMT+09:00)
UPDATE_IN_PROGRESS amplify-text-from-image-dev-233955 AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:25 GMT+0900 (GMT+09:00) User Initiated
⠸ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UpdateRolesWithIDPFunctionRole AWS::IAM::Role             Tue Jan 21 2020 00:27:30 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS authtextfromimaged2d3ad30      AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:30 GMT+0900 (GMT+09:00)
⠏ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS authtextfromimaged2d3ad30       AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:32 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS UpdateRolesWithIDPFunctionRole  AWS::IAM::Role             Tue Jan 21 2020 00:27:31 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS predictionsidentifyText6d7bda0d AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:31 GMT+0900 (GMT+09:00) Resource creation Initiated
⠙ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS amplify-text-from-image-dev-233955-authtextfromimaged2d3ad30-XXXXXXXXXXX AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:32 GMT+0900 (GMT+09:00) User Initiated
⠹ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS IdentifyTextPolicy                                                               AWS::IAM::Policy           Tue Jan 21 2020 00:27:35 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS amplify-text-from-image-dev-233955-predictionsidentifyText6d7bda0d-XXXXXXXX AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:31 GMT+0900 (GMT+09:00) User Initiated
⠹ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS SNSRole AWS::IAM::Role Tue Jan 21 2020 00:27:39 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS SNSRole AWS::IAM::Role Tue Jan 21 2020 00:27:39 GMT+0900 (GMT+09:00)


CREATE_IN_PROGRESS IdentifyTextPolicy AWS::IAM::Policy Tue Jan 21 2020 00:27:37 GMT+0900 (GMT+09:00) Resource creation Initiated
⠴ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE UpdateRolesWithIDPFunctionRole AWS::IAM::Role Tue Jan 21 2020 00:27:50 GMT+0900 (GMT+09:00)
⠴ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE IdentifyTextPolicy AWS::IAM::Policy Tue Jan 21 2020 00:27:54 GMT+0900 (GMT+09:00)
⠙ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE amplify-text-from-image-dev-233955-predictionsidentifyText6d7bda0d-XXXXXXXXX AWS::CloudFormation::Stack Tue Jan 21 2020 00:27:56 GMT+0900 (GMT+09:00)
⠙ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE SNSRole AWS::IAM::Role Tue Jan 21 2020 00:28:00 GMT+0900 (GMT+09:00)
⠋ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE predictionsidentifyText6d7bda0d AWS::CloudFormation::Stack Tue Jan 21 2020 00:28:05 GMT+0900 (GMT+09:00)
⠹ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPool AWS::Cognito::UserPool Tue Jan 21 2020 00:28:04 GMT+0900 (GMT+09:00)
⠸ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClient AWS::Cognito::UserPoolClient Tue Jan 21 2020 00:28:10 GMT+0900 (GMT+09:00)
CREATE_COMPLETE    UserPool       AWS::Cognito::UserPool       Tue Jan 21 2020 00:28:07 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS UserPool       AWS::Cognito::UserPool       Tue Jan 21 2020 00:28:07 GMT+0900 (GMT+09:00) Resource creation Initiated
⠏ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientWeb AWS::Cognito::UserPoolClient Tue Jan 21 2020 00:28:11 GMT+0900 (GMT+09:00)
⠋ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE    UserPoolClientWeb AWS::Cognito::UserPoolClient Tue Jan 21 2020 00:28:13 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS UserPoolClientWeb AWS::Cognito::UserPoolClient Tue Jan 21 2020 00:28:13 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_COMPLETE    UserPoolClient    AWS::Cognito::UserPoolClient Tue Jan 21 2020 00:28:13 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS UserPoolClient    AWS::Cognito::UserPoolClient Tue Jan 21 2020 00:28:13 GMT+0900 (GMT+09:00) Resource creation Initiated
⠹ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientRole AWS::IAM::Role Tue Jan 21 2020 00:28:17 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS UserPoolClientRole AWS::IAM::Role Tue Jan 21 2020 00:28:16 GMT+0900 (GMT+09:00)
⠇ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE UserPoolClientRole AWS::IAM::Role Tue Jan 21 2020 00:28:36 GMT+0900 (GMT+09:00)
⠹ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientLambda AWS::Lambda::Function Tue Jan 21 2020 00:28:40 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS UserPoolClientLambda AWS::Lambda::Function Tue Jan 21 2020 00:28:40 GMT+0900 (GMT+09:00)
⠏ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE UserPoolClientLambda AWS::Lambda::Function Tue Jan 21 2020 00:28:41 GMT+0900 (GMT+09:00)
⠸ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientLambdaPolicy AWS::IAM::Policy Tue Jan 21 2020 00:28:45 GMT+0900 (GMT+09:00)
⠦ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientLambdaPolicy AWS::IAM::Policy Tue Jan 21 2020 00:28:46 GMT+0900 (GMT+09:00) Resource creation Initiated
⠸ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE UserPoolClientLambdaPolicy AWS::IAM::Policy Tue Jan 21 2020 00:29:03 GMT+0900 (GMT+09:00)
⠴ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientLogPolicy AWS::IAM::Policy Tue Jan 21 2020 00:29:09 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS UserPoolClientLogPolicy AWS::IAM::Policy Tue Jan 21 2020 00:29:08 GMT+0900 (GMT+09:00)
⠴ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UserPoolClientInputs    Custom::LambdaCallout Tue Jan 21 2020 00:29:30 GMT+0900 (GMT+09:00)
CREATE_COMPLETE    UserPoolClientLogPolicy AWS::IAM::Policy      Tue Jan 21 2020 00:29:26 GMT+0900 (GMT+09:00)
⠴ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE    UserPoolClientInputs Custom::LambdaCallout Tue Jan 21 2020 00:29:34 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS UserPoolClientInputs Custom::LambdaCallout Tue Jan 21 2020 00:29:34 GMT+0900 (GMT+09:00) Resource creation Initiated
⠦ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS IdentityPool AWS::Cognito::IdentityPool Tue Jan 21 2020 00:29:40 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS IdentityPool AWS::Cognito::IdentityPool Tue Jan 21 2020 00:29:38 GMT+0900 (GMT+09:00)
⠼ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS IdentityPoolRoleMap AWS::Cognito::IdentityPoolRoleAttachment Tue Jan 21 2020 00:29:45 GMT+0900 (GMT+09:00)
CREATE_COMPLETE    IdentityPool        AWS::Cognito::IdentityPool               Tue Jan 21 2020 00:29:41 GMT+0900 (GMT+09:00)
⠴ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE    amplify-text-from-image-dev-233955-authtextfromimaged2d3ad30-XXXXXXX AWS::CloudFormation::Stack               Tue Jan 21 2020 00:29:49 GMT+0900 (GMT+09:00)
CREATE_COMPLETE    IdentityPoolRoleMap                                                        AWS::Cognito::IdentityPoolRoleAttachment Tue Jan 21 2020 00:29:46 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS IdentityPoolRoleMap                                                        AWS::Cognito::IdentityPoolRoleAttachment Tue Jan 21 2020 00:29:46 GMT+0900 (GMT+09:00) Resource creation Initiated
⠼ Updating resources in the cloud. This may take a few minutes...

CREATE_COMPLETE    UpdateRolesWithIDPFunction AWS::Lambda::Function      Tue Jan 21 2020 00:30:14 GMT+0900 (GMT+09:00)
CREATE_IN_PROGRESS UpdateRolesWithIDPFunction AWS::Lambda::Function      Tue Jan 21 2020 00:30:14 GMT+0900 (GMT+09:00) Resource creation Initiated
CREATE_IN_PROGRESS UpdateRolesWithIDPFunction AWS::Lambda::Function      Tue Jan 21 2020 00:30:13 GMT+0900 (GMT+09:00)
CREATE_COMPLETE    authtextfromimaged2d3ad30  AWS::CloudFormation::Stack Tue Jan 21 2020 00:30:11 GMT+0900 (GMT+09:00)
⠦ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UpdateRolesWithIDPFunctionOutputs Custom::LambdaCallout Tue Jan 21 2020 00:30:17 GMT+0900 (GMT+09:00)
⠇ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS UpdateRolesWithIDPFunctionOutputs Custom::LambdaCallout Tue Jan 21 2020 00:30:20 GMT+0900 (GMT+09:00) Resource creation Initiated
⠸ Updating resources in the cloud. This may take a few minutes...

UPDATE_COMPLETE                     amplify-text-from-image-dev-233955 AWS::CloudFormation::Stack Tue Jan 21 2020 00:30:25 GMT+0900 (GMT+09:00)
UPDATE_COMPLETE_CLEANUP_IN_PROGRESS amplify-text-from-image-dev-233955 AWS::CloudFormation::Stack Tue Jan 21 2020 00:30:24 GMT+0900 (GMT+09:00)
CREATE_COMPLETE                     UpdateRolesWithIDPFunctionOutputs  Custom::LambdaCallout      Tue Jan 21 2020 00:30:21 GMT+0900 (GMT+09:00)
✔ All resources are updated in the cloud

これでAWS側の作業は終わりです。

実装

AngularのAmplify設定

AngularでAmplifyを動かすための設定をします

"compilerOptions": {
    "types" : ["node"]
}
(window as any).global = window;
(window as any).process = {
  env: { DEBUG: undefined },
};
 
global.Buffer = global.Buffer || require('buffer').Buffer;

テキスト識別の実装

Angularを使っているとObservableに変換したいので、fromを使って変換しています。

import { Component } from '@angular/core';
import Amplify from 'aws-amplify';
import awsconfig from 'src/aws-exports';

import { AmazonAIPredictionsProvider } from '@aws-amplify/predictions';
import Predictions from '@aws-amplify/predictions';
import { from } from 'rxjs';
import { map } from 'rxjs/operators';
Amplify.configure(awsconfig);
Amplify.addPluggable(new AmazonAIPredictionsProvider());

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.scss']
})
export class AppComponent {
  readText$;

  onChangeInput(files: FileList) {
    console.log(files);
    const file = files.item(0);
    this.identify(file);
  }

  identify(file: File) {
    const data$ = from(
      Predictions.identify({
        text: { source: { file }, format: 'PLAIN' }
      })
    );

    this.readText$ = data$.pipe(map(data => data.text.fullText));
  }
}

async pipeを使ってデータを出力しています。

<h3>Text identification</h3>
<input type="file" (change)="onChangeInput($event.target.files)" />
<ng-container *ngIf="readText$ | async as readText">
  <p>{{ readText }}</p>
</ng-container>

動作確認

ローカルで立ち上げて動作確認してみましょう。

$ ng serve

http://localhost:4200にアクセスして、画像をアップロードしたら、その画像に含まれる文字が表示されたら成功です。

とりあえず、試したい人は下記URLに公開しているので、そちらから試すことができます。

https://dev7699.d2ox6yvc4fqafc.amplifyapp.com/

また、コードはこちらから確認することができます。

さいごに

Amplify+Angular+Recognitionを使って画像からテキストを読み取るアプリケーションを作ってみました。

Amplify CLIを使うことでマネージメントコンソールを開くことなく、対話形式でCFnテンプレートを作成でき簡単にAWSリソースを構築することができます。

ただ、慣れないうちはどんなリソースが作れているのか不安になってきます。はじめは都度、リソースを確認しどんなリソースが作成されるのか把握するのが良いかと思います。

慣れてくると、とても便利で手放せなくなってきますので是非試してみてください。

誰かの参考になれば幸いです。

参考サイト

https://dev.to/aws/full-stack-machine-learning-on-aws-reading-text-from-an-image-with-aws-amplify-amazon-rekognition-49jf