Privilege Escalation 03: How Hackers Walk Into Your Cloud
https://codykretsinger.com/episode?show=privilege-escalation&ep=3

Cold open:
Imagine you build a feature everybody builds. A little box where a user pastes a link, maybe a photo, and the app goes and fetches it. It grabs the image, shows it on the page. That's totally normal. Practically every app on Earth does this. Except, now I don't paste a photo. I paste an address that only your server can reach. A secret one inside your own cloud infrastructure. And Your server, being helpful, goes and asks it from me. So what happens next? That address I pointed your server to hands back the keys to your entire cloud account.

Not a password, not one file, the keys to everything. Your own server asks for them, but on my behalf, because I said please in the language it didn't know it spoke.

Narrator:
This is Privilege Escalation. Part of the Threat Aware Podcast Network. Incidents get reported. They rarely get explained. Each episode takes one attack apart from the inside, delivered with the perspective of someone who has spent time on both sides of an attack. How they got in, and how you stop them. Here's your host, Cody Kretsinger.

I'm Cody Kretsinger, and this is Privilege Escalation. Today is a little bit different. This is one of the ones where I answer the question nobody wants to raise their hand and ask. Today's question: what the hell is SSRF? And why does the cloud turn it from annoying into game over? There's no shame here. By the end, you'll actually get what SSRF is, and you'll never look at your cloud environment the same way. So here's the dumb question for the day because a lot of really intelligent, smart people cannot answer. This specific question. What is SSRF?

So SSRF or server side request forgery. Let's forget the name because frankly, it's terrible. What it actually is, is the ability to trick a server. So you trick that server into making a request for you, or even better, you get the server to go knock on a door that you're not allowed to go knock on. That's the whole idea. You're Not attacking the server head on per se. You're using it as your errand boy. And why would that ever work? Because apps make requests all the time. They fetch that photo you pasted. They call some other service.

They pull a preview of that link. Anytime an app reaches out and grabs something based on what a user handed it, that's a spot where maybe, just maybe, I can choose where it reaches. Give it my address instead of the normal one. Most apps never actually stop to ask. Wait, should I actually be allowed to go there? So far, this is a little mild. I'm making your server fetch stuff. It's annoying, not really critical or fatal, depending. But I want you to hold on to that for a moment because what I'm going to talk about next will make it fatal.

First, I need to tell you about the thing living inside every single cloud server because if you don't know that it's there, none of this will make sense. And most people running things in their cloud environments don't know that it is there. Picture a server that you just spun up in the cloud. It needs to do things, read a file out of storage, talk to a database. But to do any of that, it needs credentials. So how does it get them? You could hard code passwords into the server. That's a terrible idea, but anybody who's ever leaked a secret did it that way.

So, cloud providers solved it for you. Every server gets a little helper sitting at one fixed IP address, and that helper hands the server its own temporary credentials automatically. No password needed. The server just asks, What am I allowed to do? And the helper says Here you go. It's called the metadata service. And honestly, it's a good idea. It's why you're not pasting passwords into code anymore. I'm not talking about a vulnerability somebody shipped by accident. This is technically infrastructure, you could call it that. It's working exactly as defined. The door that hands out the keys is technically supposed to be there.

The problem is who else can make the server walk up to that door and knock? So now that you've got both halves, let's talk about what happens when they get together and meet. That helper, the metadata service we just discussed, sits at the same address in every one of the big clouds. 169, 254, 169, 254. You can write it down if you want. The attackers already have it memorized. It is ingrained in this brain. Its job is to answer the server's question about itself. What's my config? What am I allowed to do? And this is the cool one.

Here are my credentials. The temporary keys the server uses to walk into the rest of the clouds. It just hands those out to anything that asked from that server. No password by design. That is SSRF. Which means that I can make your server ask for things. So I point your server at that address. Your server, helpful as ever, walks up and knocks. The helper doesn't know I'm the one who sent it. It just sees the server asking, which is completely normal. So what does it do? It hands the keys over. They come back through your app to me.

Now I'm holding valid credentials for your cloud account. Stolen off of a laptop, not fished, just politely handed to me through your own application, like it was a feature. And in a way it was. That's the three-step trick. The first one, find a spot where the app fetches what I tell it to. Two, aim it at an address that the app was never meant to really reach. And then three, catch the keys on the way back. That's it. That's the distance between weird little bug in a photo uploader and I own your cloud account. Okay.

They've got the keys. What does it actually mean? Because credentials is a little abstract here. I want you to really understand this one. When I hold the server's keys, I can do anything the server was allowed to. I am the server now for all intents and purposes. So the real question, the one that decides Really, how bad your day gets is was that server ever allowed to do anything? When somebody sets servers up and things don't work, the fastest fix that people do very, very often is give it more access. Can it read the file?

Nope. Grant it more permissions. Is that still broken? Grant even more. Nobody ever comes back later and trims this down so that. Photo preview server you forgot about. Now there's a real chance it can read storage buckets or reach a database or maybe spin up a new machine and it's all on your bill. And maybe it can read other secrets, which lets me jump to the next thing and then the next. That's the one part that turns one bug into a whole breach. I don't stop at the server. I use its keys to go shopping.

And in a lot of cloud environments, one over permission server is all it takes to end up somewhere you'd never let a stranger near. This is exactly how a hundred million credit card applications walked out of Capital One in 2019. One request an outsider got to make, one metadata service that handed over the keys, and one role that could read all of the storage. That was a very, very bad day. Now, let me tell you about the other side. When I'm poking at a cloud app, for example, and I find an SSRF, the second I confirm I can make the server fetch a URL control, well, that's it.

That's a good day for me as an attacker. I already know how the rest of the story ends because I've just described it to you. It happens like that every time. I know this address by heart. Every single bad guy does too. It's one of the first things you reach for. You get an SSRF. You aim it straight at the metadata service and you hold your breath for about half a second, and then the credentials scroll up on your screen. And then from that moment, I'm not some guy poking at a web app anymore. I'm the server.

And I've got its keys. And from an attacker's perspective, that's where the fun starts. Finding out what the server was trusted to touch and walking it as far as it'll go. That should bother you. None of that was hard. And I'm not a savant, I promise you that. I just know the door exists and was there, and the people that built the app didn't. That really is the whole gap in this style of attack. That's the entire edge. I just read one more page of the quote unquote manual than they did. So here's a fair question.

If this is so well known, if every attacker has that address tattooed in their brain, ingrained in their head, why is it still everywhere? And there's a couple of honest reasons. For years, the old wide open version of that helper was just the default. Almost like a Microsoft default. You spun up a server, you got the version that hands the keys to everybody who asks, and nobody told you that there was a safer one. So it's baked into millions of machines that are just out there doing their thing right now. Two, turning the safe version on can actually break a lot of stuff.

And That's not great because if it depends on the old way and it might break production, that's where fixes go to die. And the last one, number three, this is a big one, is because nobody's identified it to be their job. The person who could flip this switch usually doesn't know that it exists. And the person who knows it exists can't actually reach the switch itself. So it just sits there for years, working exactly as it was designed and just waiting for the thing to happen. Now for the defender side and good news because there's actually really good fixes, and this is rare that there's a really decent fix.

So enjoy this for a moment. This is the big one. There's a newer version of the metadata helper that shuts this all down. The old version hands keys to anybody who asks, whereas the new one makes the server prove the request actually came from itself in a way that an SSRF can't fake. In the big clouds, it's called IMDS version two. If you can turn it on, turn it on. And what I mean by that is enforce it. Not available, enforced everywhere. So that the wide open version is flat out refused. And if you don't know whether you're enforcing it, now you have your homework for tonight.

Next, lockdown where your servers are even allowed to reach. Most of them have absolutely no business talking to Random internal addresses. So don't let them give every server the least amount of access it actually needs. So even if I grab its keys, they barely open anything. That alone would have shrunk half the breaches I just described. And validate anything your app fetches on a user's behalf. So I just can't hand it to my address and watch it obey. None of that is exotic. It's just work that nobody assigns. Here's my honest take, and this is gonna sting.

You don't have a cloud security problem. You have a nobody here can actually explain how the cloud works problem. We moved everything into the cloud years ago. We told ourselves that it was more secure and plenty of other things. And sometimes that's true, but more secure, quietly shipped with a secret address that hands out the keys to your whole account to anyone who can trick one server into asking. And most of the people. Running production in the cloud right now have never heard of it. They can't tell you the IP. They don't know what version they're running.

It's not their fault. Nobody taught them, but it is a problem. We got really good at spinning things up. We never got good at understanding what we spun up. We clicked the button, it worked, we moved on, and every one of those buttons quietly wired up a server with keys sitting next to a door that hands them out. And the attackers We understand it just fine. We've been reading the manual the whole time. You were just busy shipping a new product. Here's the one takeaway. Go find out whether you're enforcing the new version of the metadata helper.

That's IMDS version two. Crossed everything you run in the cloud. Not is it available, but is it actually enforced? Everywhere with no exceptions. If you can answer that by tonight, good. If you can't, and I'm gonna hedge some bets here for a moment that a lot of you can't. That is the answer. That's my whole episode today. That's everything. The fix has existed for years. The reason it's not on isn't technology. It's that nobody who could turn it on knew that they needed to. So here's your chance. Go be the person who knew.