Configuración de Kinesis Data Streams

How do I create a Kinesis Data Stream on AWS?

To manage and process large amounts of data in real time, AWS Kinesis is one of the most effective solutions. You can create and customize a Kinesis Data Stream through the AWS console by following a few key steps. Here we take you through the creation and initial configuration process.

How to access Kinesis on AWS?

1. Log in to the AWS console.
2. Search for Kinesis in the services panel. You will find two options: Kinesis and Kinesis Video Stream. Select Kinesis to continue with Data Streams.
3. Enter the Kinesis menu and click "Get Started" to begin creating your Data Stream.

What are the options available for Kinesis?

Once inside the Kinesis environment, you will see four main options to create:

• Data Stream: Ideal for real-time processes.
• Delivery Stream: For data delivery to services such as Amazon S3.
• Analytics: Facilitates real-time analysis of streamed data.
• Video Streams: For real-time video streaming.

In our case study, we will select "Create Data Stream".

How to name and configure your Data Stream?

1. Assign a name to your stream. For example, "Platzi Kinesis".

2. Determine the number of shards. Shards define the capacity to handle the amount of traffic and records you will process.

   • Each shard can:
     • Process 1 megabyte per second of write data.
     • Support up to 5 megabytes per second for reads.

Properly assessing your data load is critical to defining how many shards you will need.

How to send data to Kinesis Data Stream?

Once the Kinesis Data Stream is configured, there are several options for sending data:

• API PUT Operation: Standard interface for data transmission.
• Kinesis Producer Library (KPL): A library that optimizes data production through a producer.
• Integration with Kinesis Firehose and Kinesis Analytics: For more advanced processing and analysis.

What is important to consider about security and monitoring?

• Data encryption: Although server-side encryption is not initially enabled, you can enable encryption with the AWS Key Management System (KMS) to protect your data.
• Data retention periods: Within a range of 24 to 168 hours, affecting the cost of the service.
• Integration with CloudWatch: Allows you to process essential metrics and logs to monitor and debug data flow.

How to customize dashboards and labels?

The monitoring section provides you with predefined dashboards to examine different metrics. Group and filter resources by tags such as "Environment" to facilitate resource management and reporting.

In summary, when creating a Kinesis Data Stream, be sure to consider the number of shards, data retention, enrollment and log management. These aspects are essential to optimize the costs and efficiency of your data streaming on AWS. While the process may seem complex initially, with practice and proper planning, you will master the creation and management of your streams to maximize the potential of your applications. Keep exploring and learning about Kinesis and other AWS services!