In a serverless data processing pipeline, which services enable an event-driven, pay-per-use processing flow?

In a serverless data processing pipeline, which services enable an event-driven, pay-per-use processing flow?

• A. AWS Lambda with EventBridge or SQS for events, and optional Glue for ETL and S3 as storage
• B. AWS Fargate with standalone processing
• C. AWS Batch
• D. Amazon EMR

Answer: (A)

Building an event-driven, pay-per-use data processing pipeline relies on a function-as-a-service compute model that can be triggered by events and scales automatically. AWS Lambda provides the on-demand compute that runs your code only when something happens, and you’re charged per invocation and per second of execution time. When events come from an event bus or a queue, such as EventBridge or SQS, they trigger Lambda functions to process data or coordinate steps in a workflow, enabling decoupled, responsive pipelines without provisioning servers. Using EventBridge or SQS to deliver events is key because it creates loose coupling between producers and consumers, allowing the system to scale and retry gracefully. S3 serves as reliable, scalable storage for input and output data in the pipeline, with low cost and broad integration. AWS Glue adds optional, serverless ETL capability to transform or enrich data as part of the flow, again without managing infrastructure. This combination—Lambda for compute, EventBridge or SQS for eventing, S3 for storage, and Glue for ETL when needed—delivers a true pay-per-use, serverless data processing flow that can scale automatically with workload and minimize operational overhead. In contrast, other options rely on containers or clusters (Fargate with standalone processing), batch-oriented compute (AWS Batch), or long-running distributed processing engines (Amazon EMR). While these can be part of data pipelines, they’re not as naturally aligned with a purely event-driven, serverless, pay-per-use model.

Building an event-driven, pay-per-use data processing pipeline relies on a function-as-a-service compute model that can be triggered by events and scales automatically. AWS Lambda provides the on-demand compute that runs your code only when something happens, and you’re charged per invocation and per second of execution time. When events come from an event bus or a queue, such as EventBridge or SQS, they trigger Lambda functions to process data or coordinate steps in a workflow, enabling decoupled, responsive pipelines without provisioning servers.

Using EventBridge or SQS to deliver events is key because it creates loose coupling between producers and consumers, allowing the system to scale and retry gracefully. S3 serves as reliable, scalable storage for input and output data in the pipeline, with low cost and broad integration. AWS Glue adds optional, serverless ETL capability to transform or enrich data as part of the flow, again without managing infrastructure.

This combination—Lambda for compute, EventBridge or SQS for eventing, S3 for storage, and Glue for ETL when needed—delivers a true pay-per-use, serverless data processing flow that can scale automatically with workload and minimize operational overhead.

In contrast, other options rely on containers or clusters (Fargate with standalone processing), batch-oriented compute (AWS Batch), or long-running distributed processing engines (Amazon EMR). While these can be part of data pipelines, they’re not as naturally aligned with a purely event-driven, serverless, pay-per-use model.