AWS re:Invent 2023: Building cost-effective cloud architecture and AI

AWS re:Invent 2023 has been a whirlwind packed with expert insights and new product announcements, including Amazon Q and Guardrails for Amazon Bedrock. 

And the last full day hasn’t been an exception. Dr. Werner Vogels, VP and CTO of Amazon.com, delivered the final, highly anticipated keynote presentation. He gave us advice on how to build architecture with cost in mind and what AI means for technologists.

In this post, we share the highlights from his keynote presentation and give you the inside scoop you need to build efficient, impactful systems in the age of AI.

“It was all about going back to basics—a master class in how to be a great architect,” said Faye Ellis, Principal Training Architect at Pluralsight.

Want to catch up on the other keynotes? Check out our recap of AWS CEO Adam Selipsky’s keynote and AWS VP of Data and AI Dr. Swami Sivasubramanian’s presentation. 

• Did the cloud kill creativity?
• Cost and sustainability go hand in hand
• The frugal architect: Building cloud architecture with cost in mind
  • Design: Cost is a nonfunctional requirement for business
  • Design: Systems that align cost to business last
  • Design: Architecture is a series of trade-offs between functional and nonfunctional requirements
  • Measure: Unobserved systems lead to unknown costs
  • Measure: Cost-aware architectures implement cost controls
  • Optimize: Cost optimization is incremental
  • Optimize: Unchallenged success leads to assumptions
• AI predictions and insights
  • Using artificial intelligence for good
  • Reconsider when you use LLMs
  • AI will impact technologist jobs
  • Keep learning new tech skills
• New AWS services announced
• Develop AWS skills with hands-on labs

Dr. Vogels began by reminding us what life was like before cloud computing. While hardware constraints prohibited engineers from creating the architecture they wanted, they also promoted creative solutions to this problem. 

The shift to the cloud removed these constraints, and building architecture with cost in mind got lost. As Dr. Vogels said, cloud enabled us to work faster and push new products out, but the art of architecting for cost fell by the wayside in the process.

Now the pendulum is starting to swing back the other way. Organizations are becoming more interested in the cost of the systems they’re building. Creating self-imposed constraints around costs and sustainability will help them boost creativity and problem-solving.

Get the cloud transformation strategy guide.

“There’s a freight train coming your way, and you can’t escape it,” said Dr. Vogels. 

The train? Sustainability. “Technologists have a major role to play in making sure systems are as sustainable as you can use,” he said. 

When technologists build sustainable systems, they’re often also building with cost in mind. “Cost is a close proxy for sustainability,” he explained.

While re:Invent 2023 is coming to a close, our guide to the top sessions and speakers can help you make the most of the remaining days (and start planning for next year!).

So, how can you actually build architecture with cost in mind from the beginning? Dr. Vogels shared his advice on being a frugal architect and identified three main stages of frugal architecture: design, measure, and optimize.

When it comes to designing systems, things like security, compliance, and accessibility are nonfunctional, non-negotiable requirements. They don’t directly impact specific features or functions, but they’re necessary to the operation of the system. 

According to Dr. Vogels, organizations need to consider cost and sustainability as nonfunctional requirements, too. If cost isn’t part of the design process, organizations run the risk of costs outpacing revenue.

“Consider cost at every step,” said Michael Cassidy, DevOps professional and Pluralsight Author. “Making metrics visible can change spending habits and reveal what's important and what's not.”

The takeaway: You have to consider cost at every step of your design.

Tech teams shouldn’t design systems in a silo. You need to collaborate with business partners and have cost conversations. What revenue model will you use? How will you make money? Once you know that, build architectures that follow the money and match the pricing model you give to your customers.

Over time, your customers' needs may change, and your architecture probably will, too. Build architecture you can evolve without impacting customers. 

The takeaway: Business decisions and technology decisions must be in harmony. Pay off your debts.

Designing always comes with trade-offs. Improving performance, for example, will increase costs, and vice versa. Talk with your nontechnical partners to figure out the right balance between budget, performance, and business needs.

The takeaway: Align your priorities across the business.

Dr. Vogels shared an example of old houses in Amsterdam. The houses that used more energy had their meter hidden in the basement. The houses that used less energy had their meter in the hallway where they could see it every day.

In other words, if you aren’t measuring your systems, you don’t know what they’re costing you. You need to measure costs at the microservice level, costs at the system level, and overall conversion rate.

When you understand your costs at each level, it’s easier to make changes. You’ll know the cost of the change and can see its impact on conversion.

The takeaway: Define your meter; understand what you’re measuring and how it changes behavior.

Architecting with cost in mind also means creating cost controls for your organization. If something is costing too much money, you need a way to turn it off. 

Create tiers with your business partners. Tier 1 should be your most essential components; these are the pieces your application needs to work. Tier 2 components are important but less critical, and Tier 3 components are nice to have. These tiers inform the trade-offs you make.

The takeaway: Establish your tiers and determine what needs to be up and running at all times.

Tinkering sounds almost insignificant, but small changes can lead to big cost savings. Think about what you can stop, rightsize, shift, or reduce. What can you just stop? What can you move to a smaller or bigger instance? Can you reduce the amount of resources you really need? 

Figure out where your costs are going, look at the code to see what’s happening, and then fix disparities between what you expect and what you see.

The takeaway: Continuously optimize your systems and architecture.

“We’ve always done it that way” is one of the most dangerous ways of thinking. “Learn and be curious,” said Dr. Vogels.

You should always scrutinize and reevaluate the platforms and languages you use to ensure they're the right ones.

The takeaway: Disconfirm your beliefs.

Dr. Vogels also addressed the tech on everyone’s minds: artificial intelligence (AI) and machine learning (ML). Here are some of his insights.

“Technology can be a force for good,”  said Dr. Vogels. He shared the stories of organizations like Thorn, the International Rice Research Institute, and CergenX that are using AI and ML to solve real problems.

Not everything needs to be done with massive LLMs: Sometimes small, fast, and inexpensive models are right for the use case.

But not necessarily in the way you expect. You can incorporate generative AI into the apps you're building and collaborate with coding assistants to work smarter. As you work with coding assistants, the key here is to ask, adjust, and iterate. 

“AI makes predictions, professionals decide,” emphasized Dr. Vogels. “We, as humans, are the ones who make decisions.”

“It’s too much for a single person to be an expert in all things cloud,” said Dr. Vogels. Continual learning is the only way technologists will be able to keep up with the rapidly changing tech landscape.

And continually learning, whether that’s cloud or AI skills, will benefit you in the long run. “It helps you to actually accept these tools and explore new problem spaces,” he said.

Thought the product announcements were over? Not quite. AWS Solutions Architect and DevOps professional David Blocher shared his thoughts on some of the new services Dr. Vogels unveiled.

• New Amazon Inspector CI/CD capabilities: Scan container images for software vulnerabilities directly from your CI/CD pipelines with new open source plugins. “It’s a big leap in securing your container-based CI/CD pipeline ensuring vulnerabilities are caught early.”

• AWS Application Composer in VS Code: Visually compose infrastructure as code directly in Visual Studio Code. “It’s a visual playground for architecting AWS services. Drag, drop, connect, and see your infrastructure come to life directly from your IDE. It’s intuitive, efficient, and I’m personally really excited to get my hands on it.”

• Amazon SageMaker Studio Code Editor: Use a fully managed Code Editor within Amazon SageMaker. “Based on Visual Studio Code, [it] brings a familiar yet powerful space for ML development right inside SageMaker Studio.”

• myApplications in the AWS Management Console: Monitor and manage the cost, health, security posture, and performance of your applications. “A game changer in monitoring your AWS applications. It’s like mission control for your apps, offering insights on cost, performance, and security all from a single pane in the AWS console.”

• CloudWatch Application Signals: Monitor the performance of your applications with a single dashboard for all your most important application metrics, such as volume of requests, latency, and availability. “Think of it as your app’s health checkup. It automatically instruments EKS applications, providing key performance metrics for visualization and monitoring.”

re:Invent 2023 may be coming to an end, but your AWS journey is just beginning. If you’re ready to put your skills to the test, start a free Pluralsight Skills trial and get access to skill assessments, hands-on labs, and thousands of learning paths. These AWS labs will get you started: 

• Hands-on labs for learning AWS essentials

• AWS security essentials

• AWS developer essentials

If you’re looking to develop cloud and AI skills across your organization, check out our cloud transformation solution and AI training.