tl;dr
I’d put this at a beginner to intermediate level. If you’re just getting your hands dirty with AI, this is probably the right place to start. If you already have hands-on experience, or you’ve built AI systems at the enterprise level, this may not be the course for you, and it definitely isn’t if you want to go deeper on attacking or defending these systems.
whoami
My name is Tayvion. I’ve spent roughly eight years in security, most of it in detection engineering and cloud, and I’ve been sharing what I figure out along the way for about ten, though never on any real schedule. The work has mostly sat on the defensive side, but it has naturally required me to do a fair amount of ad hoc testing and validation. I wouldn’t necessarily call that red teaming or pentesting. It’s the testing you end up doing to support a detection you’re writing or an architecture decision you’re about to make. There’s more about me on my About page.
what’s my experience with ai
For the last year and some change my experience with AI has gone from small personal projects for home automation and mundane tasks, to implementing AI systems at the enterprise level across the detection and response function. That work goes past alert summarization into entire harnesses, and validating that those systems run as intended and run securely. I’ve built on SDKs from Anthropic, Google, and OpenAI, along with some open-weight models. So I would say I’m extremely comfortable leveraging LLMs and agentic AI systems.
Most of the projects I’ve completed haven’t been published, because of an extremely busy schedule and life lifing, but I’ve decided to start publishing the things that grab my interest. Right now that’s a series working out whether an attacker who can write to cloud state can steer a security agent that reads that state as evidence. You can read the intro to the series here.
I came into this course with a decent amount of real-world experience, so I didn’t learn much from it. Someone with limited AI experience would get a lot more out of it. This is a good course if you’re looking to get started on how LLMs work and the basics of attacking and defending them.
the platform
The first thing that caught my attention was how well the platform is built and integrated together. I’d say this is one of the better platforms I’ve used for training. A lot of courses now run their support through Discord or some other communications tool, and they’ve decided to build it directly into the platform instead. Most of the questions I needed to ask got answered almost instantly, because they’ve documented and anticipated a lot of what you’re going to run into. The support obviously uses AI to fetch those answers, but it’s a great addition, since you don’t have to go searching around the website. And if you need actual human support, they’re quite fast responding over chat too.
the labs
This is where I had mixed feelings. There were things I liked, things I felt mixed about, and things I did not like. Let me start with what I liked.
None of the labs required any additional setup. They host everything on their platform, and they’ve documented the labs extremely well. They anticipated most of the issues you would run into and already had warnings or troubleshooting steps sitting there for them. They also did a good job of reinforcing what you learned in earlier labs inside the later ones. I think that was a really smart idea, because it makes sure the person taking the course is actually retaining the information.
There’s a lot of Python across the labs, and the course is straightforward about the fact that knowing how to code isn’t the objective. Interacting with the LLM is the main goal. That one left me with mixed feelings. You do have the freedom to experiment with the code on your own, but I wish the labs were more open ended and vague, to the point where you had to figure out how to build certain things yourself. What better way to truly understand the model than having to build it? For the audience this is aimed at, though, I think it may have been the right call. And even though they never push you to know Python, they do a good job explaining what each library is for, why it’s there, and how it’s being used.
The labs are mostly guided, with some challenges thrown in for you to figure things out. Most of them also close automatically after an hour of use, so if you’re the kind of person who tinkers, you’ll more than likely have to do your deeper experimenting on your own compute.
The one thing I absolutely did not like is having to copy and paste the commands. I like to type things out, for the repetition and the muscle memory. And if you catch yourself trying to complete a lab without having pasted every command into the terminal, you have to go back to the step you missed. It doesn’t tell you which command you missed, so you end up going back through and pasting again just to make sure you actually completed it.
where it stops
This course deals with LLMs and chatbots, and we are so far past this part of AI. The attack surface has expanded a lot. We are now leveraging agentic systems, tool calls, permissions, and more. When the course covered prompt injection, the examples came from GPT-3.5-era models and DAN jailbreaks.
I wouldn’t rush to call it outdated, because fundamentals are important. But with how fast AI is moving, a lot of what’s outlined here may no longer be relevant. That’s a common thing with most training, and especially with AI, where there’s a new milestone or advancement every few months. So this is a classic case of chasing your own tail.
There were a few labs where, instead of using a plain LLM, I built out an agentic orchestration and tested the attacks against that.
MCP servers are barely discussed outside of a quick intro. However, Practical DevSecOps does have a course dedicated to this called CMCPSE, which I’m enrolled in now and learning a lot from, so it’ll get its own review once I’m through it.
And with all the shortcomings I felt this course had, it looks like they’re releasing the Certified Agentic AI Security Expert (CAASE) late this year, which should cover what this one was missing.
who this is for
If you’re coming into AI security without much background, CAISP does the job it sets out to do. The platform stays out of your way, and you’ll come out with a working understanding of how these systems get attacked and defended.
If you already work in this space and came looking for complexity, you’ll want more than this course gives you. It’s a floor, and floors are useful. The agentic material only makes sense once you have the fundamentals this course covers, and building those fundamentals is the part it gets right.
The exam is still ahead of me. I’ll come back with that verdict when I have it.