Before configuring the Kinesis stream, you should create and configure the BDS Bucket where BDS will write event objects if they can’t be written to the Kinesis stream. You should also create and configure the SNS topic where BDS will write notifications if events cannot be written to the stream.
Configuring the Kinesis Stream¶
To configure the Kinesis stream, you should download and install the AWS Command Line Interface software if you have not done so already.
Later in this web page is a YAML configuration template for setting up a BDS Kinesis stream. The file contains a Parameters section listing values you must specify during the configuration process. Each parameter has a description explaining what it is and what type of value should be specified. When you perform the configuration, you should be prepared to specify these values.
One of the configuration parameters is BrightspaceAccountID. This designates the Brightspace entity that will be given permission to write to the Kinesis stream. When you enroll for Brightspace Data Streams, you will be told the value you must specify for this parameter.
The configuration template below also contains a list of output values that will be produced by the configuration process. When you perform the configuration, record the output values and send them to D2L so that BDS has the information it needs in order to write to the stream.
AWS offers several ways to submit the template in order to configure the Kinesis stream:
Through the AWS management console
Through the AWS command line interface
Through the AWS API
Configuration File¶
The recommended YAML configuration file for the Kinesis stream is:
# Copyright 2019 D2L Corporation
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
AWSTemplateFormatVersion: 2010-09-09
Description: >
Sample template for deploying a data stream built on Amazon Kinesis with a
role that another account can assume to write to the stream.
Parameters:
BrightspaceAccountId:
Type: String
Description: >
The identifier of the account that will publish messages *to* the data
stream built on Amazon Kinesis.
RetentionPeriodHours:
Type: Number
Default: 24
Description: >
The number of hours for the data records that are stored in shards to
remain accessible. The default value is 24. For more information about
the stream retention period, see *Changing the Data Retention Period* in
the *Amazon Kinesis Developer Guide*
(https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesis-stream.html).
ShardCount:
Type: Number
Default: 1
Description: >
The number of shards that the stream uses. For greater provisioned
throughput, increase the number of shards (see
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesis-stream.html).
Resources:
ReadPolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Resource: !GetAtt Stream.Arn
Action:
- "kinesis:Get*"
- "kinesis:DescribeStream"
- "kinesis:ListStreams"
Stream:
Type: AWS::Kinesis::Stream
Properties:
RetentionPeriodHours: !Ref RetentionPeriodHours
ShardCount: !Ref ShardCount
WritePolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Resource: !GetAtt Stream.Arn
Action:
- "kinesis:PutRecord"
- "kinesis:PutRecords"
WriteRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
AWS: !Ref BrightspaceAccountId
Action: "sts:AssumeRole"
ManagedPolicyArns:
- !Ref WritePolicy
Topic:
Type: AWS::SNS::Topic
TopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
Topics:
- !Ref Topic
PolicyDocument:
Version: 2012-10-17
Statement:
- Action: "sns:Publish"
Effect: Allow
Principal:
AWS: "*"
Resource: !Ref Topic
Condition:
StringEquals:
AWS:SourceOwner: !Ref BrightspaceAccountId
Outputs:
ReadPolicyArn:
Description: >
The ARN of the IAM policy that the function built with AWS Lambda will
assume to read events to a customer data stream built on Amazon Kinesis.
Value: !Ref ReadPolicy
StreamArn:
Description: >
The ARN of the data stream built on Amazon Kinesis that Brightspace
events will be written to.
Value: !GetAtt Stream.Arn
StreamName:
Description: >
The name of the data stream built on Amazon Kinesis that Brightspace
events will be written to.
Value: !Ref Stream
WriteRoleArn:
Description: >
The ARN of the IAM role that the function built with AWS Lambda will
assume to write events to a customer data stream built on Amazon
Kinesis.
Value: !GetAtt WriteRole.Arn
WriteRoleName:
Description: >
The name of the IAM role that the function built with AWS Lambda will
assume to write events to a customer data stream built on Amazon Kinesis.
Value: !Ref WriteRole
TopicArn:
Description: >
The ARN of the sns topic that Brightspace will notify on when events fail to ship
Value: !Ref Topic