Stay ahead of the tech that matters: Attend in-person QCon London (April 4-6, 2022), or online QCon Plus (May 10-20, 2022). Register Now
Facilitating the spread of knowledge and innovation in professional software development


The Continuous Documentation methodology is a useful paradigm that helps ensure that high-quality documentation is created, maintained, and readily available. Code Walkthroughs take the reader on a “walk” — visiting at least two stations in the code — describe flows and interactions, and often incorporate code snippets.
Charles Humble discusses the design of the Ballerina programming language with its lead designer James Clark. They discuss how the goals of the language inform a number of design choices including: the type system, error handling, the concurrency model, and the language’s built in support for visualization of program flows.
In the podcast, Meenakshi Kaushik and Neelima Mukiri from the Cisco team speak on responsible AI and machine learning bias and how to address the biases when using ML in our applications.
Redgate Software runs a yearly deliberate reteaming process across engineering to alter how they invest the efforts of teams and encourage people to move towards the work they find most engaging. Self-selection reteaming is an effective and empowering method of aligning with company goals. It normalized the idea of people moving between teams for personal development and renewed sense of purpose.
Tammy Bryant Butow covers practical lessons learned in the SRE Apprentices program, things she’d change and shares how to create and roll out such a program.
Learn from practitioners driving innovation and change in software. Attend in-person on April 4-6, 2022.
Your monthly guide to all the topics, technologies and techniques that every professional needs to know about. Subscribe for free.
Uncover emerging trends and practices from software leaders. Attend online on May 10-20, 2022.
InfoQ Homepage News AWS Step Functions Gains Integrations With Over 200 Additional Services
Nov 30, 2021 4 min read
by
Steffen Opel
Amazon Web Services (AWS) recently announced that AWS Step Functions supports over 200 AWS services via an AWS SDK integration, thereby expanding the number of supported API actions from 46 to over 9,000. The new integration type can support most existing and future services and API actions going forward.
AWS Step Functions is a "low-code visual workflow service" used to orchestrate AWS services, automate business processes, and build serverless applications. The service offers two distinct workflow types for use cases that target either auditable and potentially long-running workflows (up to a year) or high-volume event processing workloads (up to five minutes). Typical workflow orchestration aspects like "failures, retries, parallelization, service integrations, and observability" are fully managed so that "developers can focus on higher-value business logic" (previous coverage).
So far, Step Functions offered now so-called optimized integrations with 17 AWS services. Former AWS Distinguished Engineer Tim Bray, who "did a whole lot of work" on the service's creation, explains that 'optimized' means "it's smart about the way it calls the service and can operate in either fire-and-forget or wait-for-completion mode [and] can autogenerate IAM policies to make your life easier". This more ambitious developer experience resulted in the list of supported AWS services growing comparatively slowly, with the ability to send custom events to Amazon EventBridge being the latest integration published in May 2021.
To remove the added cost and complexity for working around unavailable integrations with custom code invoked via the ubiquitous AWS Lambda integration, Step Functions now provides a generic AWS SDK integration that can support most services and API actions going forward, with exceptions "for a variety of reasons". This flexibility comes at the expense of having to manually configure the otherwise autogenerated IAM policies. The Amazon States Language (ASL) syntax for the new feature follows the well-established ARN based format for URI-based integration:
In a popular tweet, Tim Bray summarized the expected impact of this "no-code invocation, from Step Functions, of any API defined in the AWS SDK":
Hear that screaming? That’s the sound of a huge number of utility-glue Lambda functions realizing they are now superfluous.
In the launch blog post, AWS Senior Developer Advocate Marcia Villalba walks through a comprehensive workflow example that uses several SDK integrations to fetch a media file from an S3 bucket, then transcribes the video, and finally translates the transcript into the target language.
AWS Step Functions SDK integration example
Image: AWS Step Functions SDK integration example (via launch blog post)
Notably, the transcription job is an asynchronous operation that requires a polling loop before moving to the next workflow state. While polling loops are straight forward to implement, AWS Community Hero Ben Kehoe cautioned that such use cases would still benefit from optimized integrations:
Right, but while the AWS SDK integration allows for "you can do it today", it should not be an excuse for Step Functions not to build sync integrations! Polling loops in state machines are serious undifferentiated heavy lifting.
On a related note, AWS Community Hero Ben Bridts wishes "pagination support for Step Functions SDK integrations" to deal with larger API responses:
Building pagination in the Amazon States Language is giving me a headache, and the SDK already knows how to do it […] Bonus-points for also supporting SDK-based waiters.
Some of these limitations can be worked around by higher level tooling. For example, the Cloud Development Kit (CDK) support for the AWS SDK integration pattern assists with IAM policy generation by automatically adding an IAM statement to the state machine role's policy "based on the service and action called" and only requires intervention in the few cases "where the IAM action name does not match with the API service/action name":
In related news, AWS Athena and AWS Batch have integrated AWS Step Functions into their service consoles to visualize workflows with contextual deep links to queries and jobs, and to ease getting started via sample projects for common use cases.
Microsoft's Azure Logic Apps service uses a different extensibility architecture based on connectors and provides many built-in connectors for other Azure services, plus the ability to build a custom connector for more tailored scenarios. Google Cloud Platform's Workflows also publishes dedicated connectors to access other Google Cloud products.
The AWS Step Functions documentation features a developer guide, including a section on using Step Functions with other services, and the API reference. The AWS CLI Step Functions commands are documented separately. Support is provided via the AWS Step Functions forum. AWS SDK integrations are provided at no additional charge beyond the regular usage-based pricing for Step Functions and each target service.

Design, automate and improve processes to provide better customer experiences, deliver projects faster and increase business agility. Try now!
A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.
You need to Register an InfoQ account or or login to post comments. But there’s so much more behind being registered.
Get the most out of the InfoQ experience.
Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.
QCon, the international software development conference, is returning (in-person and online) in Spring 2022.
QCon brings together the world’s most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices.
Find practical inspiration (not product pitches) from software leaders deep in the trenches creating software, scaling architectures and fine-tuning their technical leadership to help you make the right decisions.
Attend in person at QCon London, (April 4-6) or online at QCon Plus (May 10-20). Save your spot now!
InfoQ.com and all content copyright © 2006-2021 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we’ve ever worked with.
Privacy Notice, Terms And Conditions, Cookie Policy

source

Leave a Reply