Skip to content

Commit 56156b8

Browse files
aburkleaux-amazonaburkleaux-amazon
authored
Merge MIE 0.1.4 from Development (#122)
Deliver MIE version 0.1.4 from development This merge includes several changes that improve the first user experience. These changes include: link Help menu to Implementation Guide Rename the cognito app client for the webapp so it's easier to understand which app client should be used for boto3 and which should be used for Amplify. clear canvas if user clicked the label button a second consecutive time advise user to "Try lowering confidence threshold" when elasticsearch returns no data prevent bounding boxes from overlapping Persist the workflow execution history on the upload page. add a hyperlink to workflow status for accessing step function execution details add line break between workflow config and execution history indicate when a thumbnail image is not available allow users to control them thumbnail seek position in workflow config alphabetize the transcribeLanguages list Push all assets in parallel to the collection table so the table updates in O(1) instead of O(n) time. Show both date and time in Created column in Collection view Add operator for thumbnail creation and remove thumbnail creation from the mediaconvert (transcribe) operator. fix paging bug By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
1 parent cf8901e commit 56156b8

29 files changed

+1210
-367
lines changed

‎IMPLEMENTATION_GUIDE.md

+446-5
Large diffs are not rendered by default.

‎README.md

+9-5
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,11 @@ MIE is a _serverless_ framework to accelerate the development of applications th
88
2. Execute workflows and store the resulting media and analysis for later use.
99
3. Query analysis extracted from media.
1010
4. Interactively explore some of the capabilities of MIE using the included content and analysis and search web application.
11-
5. Extend MIE for new applications by adding custom operators and custom data stores.
12-
11+
5. Extend MIE for new applications by adding custom operators and custom data stores.
12+
13+
# Limits
14+
15+
This preview version of MIE can support workflows on short videos up to 4 minutes in duration.
1316

1417
# Architecture Overview
1518

@@ -50,8 +53,9 @@ Deploy the demo architecture and application in your AWS account and start explo
5053

5154
Region| Launch
5255
------|-----
53-
US East (N. Virginia) | [![Launch in us-east-1](doc/images/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=mie&templateURL=https://rodeolabz-us-east-1.s3.amazonaws.com/media-insights-solution/v0.1.3/cf/media-insights-stack.template)
54-
US West (Oregon) | [![Launch in us-west-2](doc/images/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=mie&templateURL=https://rodeolabz-us-west-2.s3.amazonaws.com/media-insights-solution/v0.1.3/cf/media-insights-stack.template)
56+
US East (N. Virginia) | [![Launch in us-east-1](doc/images/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=mie&templateURL=https://rodeolabz-us-east-1.s3.amazonaws.com/media-insights-solution/v0.1.4/cf/media-insights-stack.template)
57+
US West (Oregon) | [![Launch in us-west-2](doc/images/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=mie&templateURL=https://rodeolabz-us-west-2.s3.amazonaws.com/media-insights-solution/v0.1.4/cf/media-insights-stack.template)
58+
5559

5660
The default settings for the template are configured to deploy the sample web application and all the back-end components it requires. In addition, you must set the required parameter below.
5761

@@ -90,7 +94,7 @@ After the stack successfully deploys, you can find important interface resources
9094

9195
**MediaInsightsEnginePython37Layer** is a lambda layer required to build new operator lambdas
9296

93-
**WebAppCloudfrontUrl** is the Url for the sample Media Insights web application
97+
**MediaInsightsWebAppUrl** is the Url for the sample Media Insights web application
9498

9599
**WorkflowApiEndpoint** is the endpoint for accessing the Workflow APIs to create, update, delete and execute MIE workflows.
96100

‎deployment/build-s3-dist.sh

+32-8
Original file line numberDiff line numberDiff line change
@@ -419,12 +419,36 @@ fi
419419

420420
popd
421421

422-
zip -g dist/start_media_convert.zip start_media_convert.py awsmie.py
423-
zip -g dist/get_media_convert.zip get_media_convert.py awsmie.py
422+
zip -g dist/start_media_convert.zip start_media_convert.py
423+
zip -g dist/get_media_convert.zip get_media_convert.py
424424

425425
cp "./dist/start_media_convert.zip" "$dist_dir/start_media_convert.zip"
426426
cp "./dist/get_media_convert.zip" "$dist_dir/get_media_convert.zip"
427427

428+
429+
echo "------------------------------------------------------------------------------"
430+
echo "Thumbnail Operations"
431+
echo "------------------------------------------------------------------------------"
432+
433+
echo "Building Thumbnail function"
434+
cd "$source_dir/operators/thumbnail" || exit
435+
436+
# Make lambda package
437+
[ -e dist ] && rm -r dist
438+
mkdir -p dist
439+
440+
if ! [ -d ./dist/start_thumbnail.zip ]; then
441+
zip -r9 ./dist/start_thumbnail.zip .
442+
443+
elif [ -d ./dist/start_thumbnail.zip ]; then
444+
echo "Package already present"
445+
fi
446+
447+
popd
448+
449+
zip -g dist/start_thumbnail.zip start_thumbnail.py
450+
cp "./dist/start_thumbnail.zip" "$dist_dir/start_thumbnail.zip"
451+
428452
echo "------------------------------------------------------------------------------"
429453
echo "Transcribe Operations"
430454
echo "------------------------------------------------------------------------------"
@@ -483,8 +507,8 @@ fi
483507

484508
popd
485509

486-
zip -g dist/start_transcribe.zip start_transcribe.py awsmie.py
487-
zip -g dist/get_transcribe.zip get_transcribe.py awsmie.py
510+
zip -g dist/start_transcribe.zip start_transcribe.py
511+
zip -g dist/get_transcribe.zip get_transcribe.py
488512

489513
cp "./dist/start_transcribe.zip" "$dist_dir/start_transcribe.zip"
490514
cp "./dist/get_transcribe.zip" "$dist_dir/get_transcribe.zip"
@@ -540,7 +564,7 @@ fi
540564

541565
popd
542566

543-
zip -g dist/get_captions.zip get_captions.py awsmie.py
567+
zip -g dist/get_captions.zip get_captions.py
544568

545569
cp "./dist/get_captions.zip" "$dist_dir/get_captions.zip"
546570

@@ -595,7 +619,7 @@ fi
595619

596620
popd
597621

598-
zip -g dist/start_translate.zip start_translate.py awsmie.py
622+
zip -g dist/start_translate.zip start_translate.py
599623

600624
cp "./dist/start_translate.zip" "$dist_dir/start_translate.zip"
601625

@@ -658,8 +682,8 @@ fi
658682

659683
popd
660684

661-
zip -g dist/start_polly.zip start_polly.py awsmie.py
662-
zip -g dist/get_polly.zip get_polly.py awsmie.py
685+
zip -g dist/start_polly.zip start_polly.py
686+
zip -g dist/get_polly.zip get_polly.py
663687

664688
cp "./dist/start_polly.zip" "$dist_dir/start_polly.zip"
665689
cp "./dist/get_polly.zip" "$dist_dir/get_polly.zip"

‎deployment/media-insights-stack.yaml

+11-8
Original file line numberDiff line numberDiff line change
@@ -697,7 +697,7 @@ Resources:
697697
UserPoolId: !Ref MieUserPool
698698
ExplicitAuthFlows: ['ADMIN_NO_SRP_AUTH']
699699

700-
MieAppClient:
700+
MieWebAppClient:
701701
Type: AWS::Cognito::UserPoolClient
702702
Properties:
703703
UserPoolId: !Ref MieUserPool
@@ -707,7 +707,7 @@ Resources:
707707
Properties:
708708
AllowUnauthenticatedIdentities: False
709709
CognitoIdentityProviders:
710-
- ClientId: !Ref MieAppClient
710+
- ClientId: !Ref MieWebAppClient
711711
ProviderName: !GetAtt MieUserPool.ProviderName
712712

713713
# More hacky cfn for getting the role mapping
@@ -723,7 +723,7 @@ Resources:
723723
- - 'Fn::GetAtt':
724724
- MieUserPool
725725
- ProviderName
726-
- Ref: MieAppClient
726+
- Ref: MieWebAppClient
727727

728728
CognitStandardAuthDefaultRole:
729729
Type: "AWS::IAM::Role"
@@ -1395,7 +1395,7 @@ Resources:
13951395
UserPoolId: !Ref MieUserPool
13961396
IdentityPoolId: !Ref MieIdentityPool
13971397
AwsRegion: !Ref "AWS::Region"
1398-
PoolClientId: !Ref MieAppClient
1398+
PoolClientId: !Ref MieWebAppClient
13991399

14001400
# TranscriberWebApp:
14011401
# Condition: DeployTranscriberApp
@@ -1445,10 +1445,10 @@ Outputs:
14451445
Value: !GetAtt MediaInsightsWorkflowApi.Outputs.EndpointURL
14461446
Export:
14471447
Name: !Join [":", [!Ref "AWS::StackName", WorkflowApiEndpoint]]
1448-
WebAppCloudfrontUrl:
1448+
MediaInsightsWebAppUrl:
14491449
Condition: DeployDemoSiteCondition
1450-
Description: Url of the MIE Webapp
1451-
Value: !GetAtt MediaInsightsWebApp.Outputs.CloudfrontUrl
1450+
Description: Url of the Media Insights Engine sample web application
1451+
Value: !Join ["", ["https://", !GetAtt MediaInsightsWebApp.Outputs.CloudfrontUrl]]
14521452
ElasticsearchEndpoint:
14531453
Condition: DeployAnalyticsPipelineCondition
14541454
Description: Endpoint for elasticsearch cluster
@@ -1466,8 +1466,11 @@ Outputs:
14661466
Description: ID of the MIE Cognito User Pool
14671467
Value: !Ref MieUserPool
14681468
AdminClientId:
1469-
Description: ID of the Admin Cognito Client
1469+
Description: ID of the Admin Cognito Client. This can be used to authenticate command-line apps using boto3.
14701470
Value: !Ref MieAdminClient
1471+
WebAppClientId:
1472+
Description: ID of the Webapp Cognito Client. This can be used to authenticate web apps using Amplify.
1473+
Value: !Ref MieWebAppClient
14711474
AdminUsername:
14721475
Description: Username of the default MIE admin
14731476
Value: !Ref AdminEmail

‎source/consumers/elastic/lambda_handler.py

+35-32
Original file line numberDiff line numberDiff line change
@@ -324,74 +324,76 @@ def process_face_detection(asset, workflow, results):
324324
extracted_items.append(item)
325325
bulk_index(es, asset, "face_detection", extracted_items)
326326

327-
def process_logo_detection(asset, workflow, results):
328-
# This function puts logo detection data in Elasticsearch.
329-
# The logo detection raw data was in inconsistent with Confidence and BoundingBox fields in Rekognition.
330-
# So, those fields are modified in this function, accordingly.
327+
def process_generic_data(asset, workflow, results):
328+
# This function puts generic data in Elasticsearch.
331329
metadata = json.loads(results)
332330
es = connect_es(es_endpoint)
333331
extracted_items = []
334332
# We can tell if json results are paged by checking to see if the json results are an instance of the list type.
335333
if isinstance(metadata, list):
336334
# handle paged results
337335
for page in metadata:
338-
if "Logos" in page:
339-
for item in page["Logos"]:
336+
if "Labels" in page:
337+
for item in page["Labels"]:
340338
try:
341-
item["Operator"] = "logo_detection"
339+
item["Operator"] = "generic_data_lookup"
342340
item["Workflow"] = workflow
343-
if "Logo" in item:
344-
# Flatten the inner Logo array
345-
item["Confidence"] = float(item["Logo"]["Confidence"])*100
346-
item["Name"] = item["Logo"]["Name"]
341+
if "Label" in item:
342+
# Flatten the inner Label array
343+
item["Confidence"] = float(item["Label"]["Confidence"])*100
344+
item["Name"] = item["Label"]["Name"]
347345
item["Instances"] = ''
348-
if 'Instances' in item["Logo"]:
349-
for box in item["Logo"]["Instances"]:
346+
if 'Instances' in item["Label"]:
347+
for box in item["Label"]["Instances"]:
350348
box["BoundingBox"]["Height"] = float(box["BoundingBox"]["Height"]) / 720
351349
box["BoundingBox"]["Top"] = float(box["BoundingBox"]["Top"]) / 720
352350
box["BoundingBox"]["Left"] = float(box["BoundingBox"]["Left"]) / 1280
353351
box["BoundingBox"]["Width"] = float(box["BoundingBox"]["Width"]) / 1280
354352
box["Confidence"] = float(box["Confidence"])*100
355-
item["Instances"] = item["Logo"]["Instances"]
353+
354+
item["Instances"] = item["Label"]["Instances"]
356355
item["Parents"] = ''
357-
if 'Parents' in item["Logo"]:
358-
item["Parents"] = item["Logo"]["Parents"]
356+
if 'Parents' in item["Label"]:
357+
item["Parents"] = item["Label"]["Parents"]
359358
# Delete the flattened array
360-
del item["Logo"]
359+
del item["Label"]
361360
extracted_items.append(item)
362361
except KeyError as e:
363362
print("KeyError: " + str(e))
364363
print("Item: " + json.dumps(item))
365364
else:
366365
# these results are not paged
367-
if "Logos" in metadata:
368-
for item in metadata["Logos"]:
366+
367+
if "Labels" in metadata:
368+
for item in metadata["Labels"]:
369369
try:
370-
item["Operator"] = "logo_detection"
370+
item["Operator"] = "generic_data_lookup"
371371
item["Workflow"] = workflow
372-
if "Logo" in item:
373-
# Flatten the inner Logo array
374-
item["Confidence"] = float(item["Logo"]["Confidence"])*100
375-
item["Name"] = item["Logo"]["Name"]
372+
if "Label" in item:
373+
# Flatten the inner Label array
374+
item["Confidence"] = float(item["Label"]["Confidence"])*100
375+
item["Name"] = item["Label"]["Name"]
376376
item["Instances"] = ''
377-
if 'Instances' in item["Logo"]:
378-
for box in item["Logo"]["Instances"]:
377+
if 'Instances' in item["Label"]:
378+
for box in item["Label"]["Instances"]:
379379
box["BoundingBox"]["Height"] = float(box["BoundingBox"]["Height"]) / 720
380380
box["BoundingBox"]["Top"] = float(box["BoundingBox"]["Top"]) / 720
381381
box["BoundingBox"]["Left"] = float(box["BoundingBox"]["Left"]) / 1280
382382
box["BoundingBox"]["Width"] = float(box["BoundingBox"]["Width"]) / 1280
383383
box["Confidence"] = float(box["Confidence"])*100
384-
item["Instances"] = item["Logo"]["Instances"]
384+
item["Instances"] = item["Label"]["Instances"]
385385
item["Parents"] = ''
386-
if 'Parents' in item["Logo"]:
387-
item["Parents"] = item["Logo"]["Parents"]
386+
if 'Parents' in item["Label"]:
387+
item["Parents"] = item["Label"]["Parents"]
388388
# Delete the flattened array
389-
del item["Logo"]
389+
del item["Label"]
390390
extracted_items.append(item)
391391
except KeyError as e:
392392
print("KeyError: " + str(e))
393393
print("Item: " + json.dumps(item))
394-
bulk_index(es, asset, "logos", extracted_items)
394+
395+
bulk_index(es, asset, "labels", extracted_items)
396+
395397

396398
def process_label_detection(asset, workflow, results):
397399
# Rekognition label detection puts labels on an inner array in its JSON result, but for ease of search in Elasticsearch we need those results as a top level json array. So this function does that.
@@ -674,7 +676,8 @@ def lambda_handler(event, context):
674676
if operator == "translate":
675677
process_translate(asset_id, workflow, metadata["Results"])
676678
if operator == "genericdatalookup":
677-
process_logo_detection(asset_id, workflow, metadata["Results"])
679+
process_generic_data(asset_id, workflow, metadata["Results"])
680+
678681
if operator == "labeldetection":
679682
process_label_detection(asset_id, workflow, metadata["Results"])
680683
if operator == "celebrityrecognition":

‎source/dataplaneapi/.chalice/config.json

+3-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@
1010
"dev": {
1111
"api_gateway_stage": "api",
1212
"autogen_policy": false,
13-
"iam_policy_file": "policy-dev.json"
13+
"iam_policy_file": "policy-dev.json",
14+
"lambda_timeout": 900,
15+
"lambda_memory_size": 2048
1416
}
1517
}
1618
}

‎source/operators/mediaconvert/start_media_convert.py

-36
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,6 @@ def lambda_handler(event, context):
3434

3535
file_input = "s3://" + bucket + "/" + key
3636
destination = "s3://" + bucket + "/" + 'private/assets/' + asset_id + "/workflows/" + workflow_id + "/"
37-
thumbnail_destination = "s3://" + bucket + "/" + 'private/assets/' + asset_id + "/"
3837

3938
try:
4039
response = mediaconvert.describe_endpoints()
@@ -89,41 +88,6 @@ def lambda_handler(event, context):
8988
"Destination": destination
9089
}
9190
}
92-
},
93-
{
94-
"CustomName": "thumbnail",
95-
"Name": "File Group",
96-
"Outputs": [
97-
{
98-
"ContainerSettings": {
99-
"Container": "RAW"
100-
},
101-
"VideoDescription": {
102-
"ScalingBehavior": "DEFAULT",
103-
"TimecodeInsertion": "DISABLED",
104-
"AntiAlias": "ENABLED",
105-
"Sharpness": 50,
106-
"CodecSettings": {
107-
"Codec": "FRAME_CAPTURE",
108-
"FrameCaptureSettings": {
109-
"FramerateNumerator": 1,
110-
"FramerateDenominator": 5,
111-
"MaxCaptures": 2,
112-
"Quality": 80
113-
}
114-
},
115-
"DropFrameTimecode": "ENABLED",
116-
"ColorMetadata": "INSERT"
117-
},
118-
"NameModifier": "_thumbnail"
119-
}
120-
],
121-
"OutputGroupSettings": {
122-
"Type": "FILE_GROUP_SETTINGS",
123-
"FileGroupSettings": {
124-
"Destination": thumbnail_destination
125-
}
126-
}
12791
}],
12892
"AdAvailOffset": 0,
12993
"Inputs": [{

0 commit comments

Comments
 (0)