Cisco is honored to be a companion of the Black Hat NOC (Community Operations Middle), and this was our seventh yr supporting Black Hat Asia. Cisco is the Official Cell Machine Administration, Malware Evaluation and DNS (Area Title Service) Supplier.
We work with different official suppliers to deliver the {hardware}, software program and engineers to construct and safe the community, for our joint buyer: Black Hat.
- Arista: Community Gear
- Corelight: Community Analytics and Detection
- MyRepublic: Broadband
- NetWitness: Risk Detection & Response, Id
- Palo Alto Networks: Community Safety Platform
The first mission within the NOC is community resilience. The companions additionally present built-in safety, visibility and automation, a SOC (Safety Operations Middle) contained in the NOC.
On screens outdoors the NOC have been displayed companion dashboards for the attendees to view the quantity and safety of the community visitors.
It All Began with Malware
Cisco joined the Black Hat NOC in 2016, when requested to offer automated malware evaluation with Thread Grid. The Cisco contributions to the community and safety operations advanced, with the wants of the client, to incorporate extra parts of the Cisco Safety Cloud.
The NOC leaders allowed Cisco (and the opposite NOC companions) to usher in further software program to make our inner work extra environment friendly and have higher visibility; nonetheless, Cisco isn’t the official supplier for Prolonged Detection & Response, Community Detection & Response or Collaboration.
- Breach Safety Suite
- Cisco XDR: Risk Searching / Risk Intelligence Enrichment / Government dashboards / Automation with Webex
- Cisco XDR Analytics (Previously Safe Cloud Analytics / Stealthwatch Cloud): community visitors visibility and menace detection
- Cisco Webex: Incident notification and crew collaboration
The Cisco XDR Command Middle dashboard tiles made it simple to see the standing of every of the related Cisco Safety applied sciences, and the standing of ThousandEyes brokers.
When the companions deploy to every convention, we arrange a world class community and safety operations middle in three days. Our objective stays community up time and creating higher built-in visibility and automation. Black Hat has the choose of the safety business instruments and no firm can sponsor/purchase their means into the NOC. It’s invitation solely, with the intention of variety in companions, and an expectation of full collaboration.
As a NOC crew comprised of many applied sciences and corporations, we’re constantly innovating and integrating, to offer an general SOC cybersecurity structure answer. We look ahead to persevering with the work with companion Palo Alto Networks, for additional automation at Black Hat USA 2024.
Under are the Cisco XDR integrations for Black Hat Asia, empowering analysts to analyze Indicators of Compromise (IOC) in a short time, with one search.
We recognize alphaMountain.ai, Pulsedive and Recorded Future donating full licenses to Cisco, to be used within the Black Hat Asia 2024 NOC.
An instance of that is an investigation of a doubtlessly malicious exercise on the twond day of Coaching. An IP handle was recognized by NetWitness for attainable geolocation leakage.
Investigation of the IP correlated the syslog sightings from the companion applied sciences within the NetWitness logs, with menace intelligence from Pulsedive, Recorded Future, alphaMountain and others.
Reviewing the DNS logs and the main points of the packet seize in each Corelight and NetWitness, it was confirmed no geolocation knowledge was leaked and it was a part of a Coaching course. The exercise would have been blocked in a manufacturing atmosphere.
A core built-in workflow within the Black Hat NOC is NetWitness and Corelight sending suspicious recordsdata to Safe Malware Analytics. Over 4,900 samples have been submitted.
The NOC analysts additionally used Malware Analytics to analyze suspicious domains, with out the chance of an infection. Fairly than going to the web site on a company or Black Hat belongings, we have been capable of work together with the web site within the glovebox, together with downloading and putting in the web site payload.
Detonating recordsdata or searching web sites in Safe Malware Analytics protects the analysts from unintentional an infection.
We noticed a sequence of comparable (however completely different hash values) exploit kits downloaded on the primary day within the Enterprise Corridor. The downloads have been on the convention Wi-Fi and never in a Coaching course, so the occasion needed to be investigated to substantiate there was not an assault on the attendees. Working with the Corelight crew, the NOC responders parsed the visitors and confirmed it was a Seize the Flag occasion, which continued into the final day of the convention.
Risk Hunters’ Story, by Aditya Raghavan and Shaun Coulter
Within the Black Hat Asia 2024 NOC, Shaun staffed the morning shifts, and Aditya the afternoon shifts, as menace hunters centered on the Cisco XDR and Safe Malware Analytics consoles. Mornings have been normally fairly chill. Nonetheless, and for some heretofore unknown (espresso associated?) cause, the exercise ramped up within the afternoon on most days, main Aditya to a spot of “concerned pleasure”, and Shaun to a spot of tormented jealousy :D. With dogged willpower each hunters spent their time reviewing alerts, actions, and carried out IOC scans utilizing XDR Examine. They reviewed submitted samples and community logs for indicators of intrusion or suspicious exercise.
Utilizing Safe Malware analytics, they dissected malware samples, analyzed phishing campaigns, and scrutinized community visitors patterns for anomalies. Quite a few alerts flagged as spikes in visitors from surprising sources, unusual locations and odd variants of malicious code popped up a number of occasions, initiating thorough investigations. Normally, they traced the anomaly to a certified Black Hat Coaching or Briefing supply and closed such instances as “Black Hat Optimistic”; which means you wouldn’t permit this in your manufacturing community, however for Black Hat, it’s enterprise as common. Since Black Hat is a convention designed for studying about offensive safety, these malware samples are anticipated, and marked as such.
Fortunately or unthankfully, because the system tuning was accomplished, most alerts raised have been as above and anticipated or really ‘close to misses’ – gadgets that warrant investigation however didn’t prolong to impactful behaviours, as we have been capable of cease them in time.
On the primary day of Briefings, as Shaun is dutifully poring by way of the console of Safe Malware Analytics, in walks Aditya to alleviate the shift. Greetings apart, Shaun shortly pivots over excitedly “Brother, I wish to present you a few attention-grabbing issues.” Aditya’s curiosity is piqued, and Shaun opens a brand new dashboard exhibiting one of many not too long ago launched options of Cisco XDR – MITRE ATT&CK ® Protection Map.
This new functionality shortly shows all of the techniques and strategies within the MITRE ATT&CK® matrix for which Cisco XDR has detections/protection. Along with the XDR Native, detections from Safe Endpoint and Safe Malware Analytics are additionally used to derive the protection map making it a holistic view. This view permits the person to visualise the detections of XDR natively, in addition to the built-in options and determine the scope of protection and importantly, map out the gaps for future consideration. Because of the Cisco Talos crew, all options throughout the Cisco Breach Safety Suite are mapped at the moment and this may be rolled out to incorporate different suites and options, together with 3rd celebration integrations, quickly.
As our menace hunters geek out on the behind-the-scenes stuff on XDR, Jessica politely calls out “Adi. Shaun. Guys, there’s some new exercise on Umbrella. Are you able to look into it?” Nudged again to actuality, our menace hunters get to work – discovering needles within the stack of needles at Black Hat because it was rightly put by Grifter! Speaking about that, the brand new exercise seems to be a question for a site categorized as a Command & Management (C&C) area. Let’s dig into it.
A fast look into Umbrella Exercise Search reveals the newest visitors exercise matching the C&C class that was allowed. Increasing the main points pane, we will see the area being queried and the id of the endpoint issuing the question which seems to be from the ‘Hacking Enterprises 2024 Purple Group’. That may be a reputable Coaching class at Black Hat Asia 2024. We pivot over to Umbrella Examine and see the explanation for this area being categorized as C&C and its indicators.
Let’s head over to XDR and question this observable in opposition to all of the built-in options for extra intel. We shortly get a visible related graph and tabulated occasions on all of the related intel. The mixing with NetWitness Logs gives us with occasions associated to that area, in addition to populating the graph with these relationships, together with the Umbrella occasion which was the supply for this hunt.
Trying on the proof, this turned out to be one other needle! Nothing untoward right here, we categorized this as a ‘Black Hat Optimistic’ and moved on. Because the afternoon shift winds down, the crew is discussing potential locations for dinner and there’s at all times dessert to look ahead to on the finish. Aditya and Ryan have been pining for wealthy ice cream and House Greatest Dessert seems to be the appropriate answer for the ask. Within the NOC, the appropriate answer is sort of at all times teamwork with all our companions.
One such occasion was when a Corelight hunter picked up a spike of visitors to uncommon locations. These look like DNS queries to a bunch of C&C domains. We shortly delve into Umbrella exhibiting us all of the domains being queried in a brief window and most of them being Malware and/or C&C categorized. This seems to be a system both being compromised or somebody deliberately doing a check / recon for these domains.
Let’s examine a few of these domains in XDR. We are able to see quite a lot of pink icons on this visualization! In truth, each queried area is classed as Malicious and recognized to host different malicious content material. This doesn’t look anticipated for certain and that places the intentional check / recon concept to relaxation shortly. Ben Reardon, the hunter from Corelight, places it succinctly “This field is pwned six methods to Sunday!” What else can we discover about this technique then?
Trying on the DHCP logs for the IP handle, the Corelight hunter was capable of pinpoint the machine MAC handle and hostname, which resembled a reputation. A brief Google search later, we’ve a possible machine proprietor and the truth that he was delivering a session at Black Hat in one of many rooms subsequent door! A brief dialog with the particular person after his session ensued, the place the NOC leads suggested the NOC’s findings on his compromised system. He was grateful for the discovering and reached out for extra context. This one turned out to be a ‘True Optimistic.’
The next day, the crew has zeroed in on Turkish meals for the night. Ryan halts Shaun as he departs on the finish of his shift and calls for his lodge identify and room quantity. “I’m gonna come knock at your door and wake you up tonight, man. I imply it. No day is just too lengthy. I used to do my shifts on three hours of sleep. Now, let’s go!” Ryan is deadpan critical. That’s what we thought whereas investigating our subsequent potential malware discovering.
One other occasion on the Umbrella console involves our consideration and this time it’s a question for a site categorized as Malware. The supply endpoint is shortly recognized from the Id and Umbrella examine tells us this area is a part of the Malware block record. In a standard manufacturing community, this may ideally be blocked.
Black Hat isn’t your regular manufacturing community, and it attracts all types of safety folks. And that’s precisely what it turned out to be this time. The Nationwide College of Singapore has a bunch organizing common seize the flag (CTF) occasions and is operating the same get-together at Black Hat. Go NUS Greyhats!
Actions involving malware what can be blocked on a company community should be allowed, throughout the confines of Black Hat Code of Conduct.
Community Observability with ThousandEyes, by Adam Kilgore and Patrick Yong
Deploying ThousandEyes at Black Hat is a rigorous course of involving quite a lot of {hardware} (some proven beneath), configuration, testing, troubleshooting, and operating across the convention middle.
Along with our typical deployment duties, we applied a number of enhancements to the service. These enhancements included an overhaul of the dashboards to point out granular knowledge for every convention room, alongside combination knowledge for your complete convention; and higher labeling and group of deployed brokers.
The ThousandEyes dashboard was projected on the big display within the NOC, for alerting on any community points, previous to reviews from customers.
On the troubleshooting aspect, we improved our log evaluation and assortment strategies and arrange centralized monitoring of wi-fi knowledge. These efforts contributed to enhancements in visibility and agent uptime all through the convention.
Through the preliminary two days of Coaching periods at Black Hat, ThousandEyes brokers confirmed solely minor deviations from baseline because the Coaching periods got here on-line. Because the Coaching periods continued, efficiency was steady, with solely uncommon alerts for minor degraded throughput or average latency spikes. On Thursday, all of the two-day Coaching periods have been closed, and the convention shifted in direction of Briefings, alongside two four-day Coaching periods that ran for the convention’s size. With begin of Briefings and opening the Enterprise Corridor, headcounts drastically elevated. ThousandEyes noticed degraded efficiency on the community, primarily within the massive convention rooms internet hosting the Briefings. The beneath picture reveals a check end result from the Hibiscus 3610 ballroom:
The community path above reveals heavy latency on the primary hyperlink to the default gateway, compounded by one other excessive latency hyperlink outdoors the convention community. A breakdown of connectivity for the above path is proven beneath:
The throughput quantity above is vital to this investigation. The Entry Factors (APs) for the Hibiscus 3610 ballroom had a mean throughput of round 174 Mbps. Reviewing AP logs, we discovered that 92 customers have been related to the identical AP from which the check was run. Dividing the 174 Mbps by 92 offers a mean throughput in step with the 1.7 Mbps proven above, so the poor connectivity was pushed by oversaturation of person connections on this space.
The Hibiscus 3610 room and different brokers in a close-by hallway constantly had the worst connection among the many convention rooms, as proven by our agent polling outcomes.
Whereas there have been limitations within the quantity of bandwidth out there for the convention basically, the information above suggests extra of the out there AP and bandwidth assets ought to be allotted to the Hibiscus 3610 ballroom and adjoining hallways for future convention topologies, which was shared with our Community Gear companion.
Meraki Programs Supervisor, by Paul Fidler and Connor Loughlin
Our eighth deployment of Meraki Programs Supervisor because the official Cell Units Administration platform went very easily, and we launched a brand new caching operation to replace iOS gadgets on the native community, for pace and effectivity. Going into the occasion, we deliberate for the next kinds of gadgets and functions:
- iPhone Lead Scanning Units
- iPads for Registration
- iPads for Session Scanning
We registered the gadgets upfront of the convention. Upon arrival, we turned every machine on.
Then we ensured Location Companies enabled, at all times on.
As an alternative of utilizing a mass deployment expertise, like Apple’s Automated Machine Enrollment, the iOS gadgets are “ready” utilizing Apple Configurator. This consists of importing a Wi-Fi profile to the gadgets as a part of that course of. In Las Vegas, this Wi-Fi profile wasn’t set to auto be part of the Wi-Fi, leading to the necessity to manually change this on 1,000 gadgets. Moreover, 200 gadgets weren’t reset or ready, so we had these to reimage as effectively.
Black Hat Asia was completely different. We took the teachings from Black Hat USA 2023 and coordinated with the contractor to arrange the gadgets. Now, if you happen to’ve ever used Apple Configurator, there’s a number of steps wanted to arrange a tool. Nonetheless, these will be mixed right into a Blueprint.
For Black Hat Asia this included:
- Wi-Fi profile
- Enrollment, together with supervision
- Whether or not to permit USB pairing
- Setup Assistant pane skipping
In Meraki Programs Supervisor, we managed the purposes by the assigned use, designated by Tags. After we got here in on the primary morning of the Briefings, three iPhones wanted to be modified from lead scanning within the Enterprise Corridor, to Session Scanning for the Keynote, so the attendees may fill the corridor quicker. Reconfiguring was so simple as updating the Tags on every machine. Moments later, they have been prepared for the brand new mission…which was essential because the Keynote room stuffed and needed to go to an overflow room.
We additionally have been capable of verify the bodily location of every machine if wiping was required as a result of loss or theft.
When it was time for the attendees to register, they only displayed their QR code from their private cellphone, as obtained in e mail from Black Hat. Their badge was immediately printed, with all private particulars secured.
This goes with out saying, however the iOS gadgets (Registration, Lead Seize and Session Scanning) do have entry to non-public data. To make sure the safety of the information, gadgets are wiped on the finish of the convention, which will be accomplished remotely by way of Meraki Programs Supervisor.
Content material Caching
One of many greatest issues affecting the iOS gadgets in Black Hat USA 2023 was the fast must each replace the iOS machine’s OS as a result of a patch to repair a zero-day vulnerability and to replace the Black Hat iOS app on the gadgets. There have been a whole lot of gadgets, so this was a problem for every to obtain and set up. So, I took the initiative into trying into Apple’s Content material Caching service constructed into macOS.
Now, simply to be clear, this wasn’t caching EVERYTHING… Simply Apple App retailer updates and OS updates.
That is turned on withing System Setting and begins working instantly.
I’m not going to get into the weeds of setting this up, as a result of there’s a lot to plan for. However, I’d recommend that you simply begin right here. The setting I did change was:
Location and Jailbreak detection
One factor that we haven’t spoken about in a while is Jailbreak detection and Location. There are numerous components that we get again from a tool, however two of them, Location and Jailbreak should be retrieved from a tool utilizing a supplemental utility: On this case, the Meraki Programs Supervisor agent.
HOWEVER, these can solely be retrieved from the machine if the appliance is operating within the background. If the machine has been rebooted, or the appliance terminated, then we don’t get something.
One of many different painful, however comprehensible, elements of MDM is that you would be able to’t launch an utility distant on a cell machine…. However you may!
On each Android and iOS, there’s a functionality referred to as Kiosk or Single App mode: Use instances for this are usually unattended gadgets, like in eating places, or scanning gadgets like supply drivers, and so forth. And when sending the command to the machine to enter kiosk mode will launch the appliance. You may also ship a command to take away kiosk mode from the machine too. The wonderful thing about this final level is that the appliance stays in focus and open!
So, the opposite functionality that utilizing Meraki Programs Supervisor offers us is the power to schedule settings. Subsequently, we will activate kiosk mode in the course of the evening and take away it an hour later.
To make sure that this doesn’t influence the registration employees, we will go one step additional: after we’ve launched Meraki Programs Supervisor, an hour later we will relaunch the registration utility, Swapcard Go.
Systematic ThousandEyes Agent Deployment
ThousandEyes has been a success at Black Hat. At an occasion the place understanding instantly the place points lie within the community and past to make sure a terrific convention is paramount, the visibility ThousandEyes offers is unbelievable. On condition that, and the complexity of the community right here, and on condition that we’ve a Mac Mini deployed for caching software program updates, as we’re utilizing Meraki Programs Supervisor (SM) for different functions, I believed I’d take the chance to deploy the ThousandEyes Agent utilizing SM.
The opposite cause is that, while we’ve a substantial quantity of cloud and enterprise brokers, we had no endpoint brokers deployed. Nonetheless, issues are by no means that simple with software program deployment, primarily as a result of you’ll want to provision / configure software program as soon as deployed. On cell gadgets, that is simple, both utilizing settings payloads, or by utilizing Managed Appe Config to configure an app.
On desktop, utilizing MDM, we will usually use issues like Managed Plists to do the identical factor, however the TE agent does NOT help this. As soon as put in, we should name the agent with a string.
So, to realize all this, we will package deal the agent and command right into a package deal utilizing a command line utility on the Mac referred to as PKGBUILD (extra particulars right here).
I additionally used a information I’d written for the Meraki Neighborhood, out there right here.
Information of observe:
The Postflight:
#!/bin/bash
# this identify will change with every model of the agent
installer -pkg /tmp/Endpoint Agent-x64-1.193.1.pkg -target /
/Purposes/ThousandEyes Endpoint Agent.app/Contents/MacOS/te-agent –register “YOURUNIQUESTRING”
exit 0
The command to construct the package deal utilizing PKGBUILD
Extra particulars right here or watch the video.
Repurposing of Units for the following present
We have been requested if there was something we may do to go away the gadgets as they have been for the following present. After cautious consideration, we determined that we may go away the gadgets in a state that was amenable to everybody. The foremost requirement was leaving the Swapcard Go app on the machine. However, because the app is provisioned for every present, it’s fairly the method to take away configuration after which re-add it….
So, the opposite factor to notice is the choices that we’ve when putting in (and eradicating) an utility on a managed iOS machine:
Take away with MDM is the attention-grabbing one, because it permits us to, somewhat than WIPING the machine on the finish of the present, to take away administration, together with any apps and settings, and their corresponding knowledge.
The downside with that is that this was by no means a requirement in the beginning of the present. So, we now want a course of in a selected order to facilitate this…. As that is for under a handful of gadgets:
- Deprovision the app from gadgets by unscoping the appliance in Meraki Programs Supervisor
- Wait to see this command has accomplished throughout all gadgets
- Reprovision the app utilizing MDM once more, however with this being a brand new app set up, it can permit the OS to maintain the app in situ after an unenrollment
- Wait till accomplished
- Unenroll the machine
Area Title Service Statistics, by Christian Clasen
Since 2018, we’ve been monitoring DNS stats on the Black Hat Asia conferences.
The historic DNS requests are within the chart beneath.
With over 18.2M DNS requests made, we had probably the most to this point at an Asia present. We made visibility developments on the earlier yr’s Asia convention. Previous to Asia 2023, we have been permitting attendees to make use of their chosen DNS resolvers over our assigned inner Umbrella Digital Home equipment. In coordination with Palo Alto Networks (the convention Firewall supplier), we started intercepting and redirecting DNS queries for different resolvers, to pressure decision by way of the Umbrella gear. Whereas that is solely efficient for plain-text DNS queries and never encrypted protocols like DNS over HTTPS, it never-the-less dramatically elevated visibility as evidenced by the numbers within the accompanying charts.
The Exercise quantity view from Umbrella offers a top-level degree look of actions by class, which we will drill into for deeper menace searching. On development with the earlier Black Hat Asia occasions, the highest Safety classes have been Malware and Newly Seen Domains.
In a real-world atmosphere, of the 18.2M requests that Umbrella noticed, over 2,000 of them would have been blocked by our default safety insurance policies. Nonetheless, since it is a place for studying, we usually let every part fly.
We additionally monitor the Apps utilizing DNS, utilizing App Discovery.
- 2024: 4,327 apps
- 2023: 1,162 apps
- 2022: 2,286 apps
App Discovery in Umbrella offers us a fast snapshot of the cloud apps in use on the present. Not surprisingly, Generative AI (Synthetic Intelligence) has exploded over the earlier yr as a high utility.
Umbrella additionally identifies dangerous cloud purposes. Ought to the necessity come up, we will block any utility through DNS, resembling Generative AI apps, Wi-Fi Analyzers, or the rest that has suspicious undertones.
Once more, this isn’t one thing we might usually do on our Normal Wi-Fi community, however there are exceptions. For instance, sometimes, an attendee will study a cool hack in one of many Black Hat programs or within the Arsenal lounge AND attempt to use stated hack on the convention itself. That’s clearly a ‘no-no’ and, in lots of instances, very unlawful. If issues go too far, we’ll take the suitable motion.
Through the convention NOC Report, the NOC leaders additionally report of the High Classes seen at Black Hat.
Total, we’re immensely happy with the collaborative efforts made right here at Black Hat Asia, by each the Cisco crew and all of the companions within the NOC.
Black Hat USA will probably be in August 2024, in Las Vegas. Christian Clasen will lead the Cisco crew within the NOC, so observe his weblog to see if what occurs in Vegas, stays in Vegas.
Acknowledgments
Thanks to the Cisco NOC crew:
- Cisco Safety: Christian Clasen, Shaun Coulter, Aditya Raghavan, Adam Kilgore, Patrick Yong and Ryan Maclennan
- Meraki Programs Supervisor: Paul Fidler and Connor Loughlin
- Further Help and Experience: Adi Sankar, Robert Harris, Jordan Chapian, Junsong Zhao, Vadim Ivlev and Ajit Thyagarajan
Additionally, to our NOC companions NetWitness (particularly Iain Davidson and Alessandro Zatti), Palo Alto Networks (particularly James Holland and Jason Reverri), Corelight (particularly Mark Overholser and Eldon Koyle), Arista Networks (particularly Jonathan Smith), MyRepublic and your complete Black Hat / Informa Tech employees (particularly Grifter ‘Neil Wyler’, Bart Stump, Steve Fink, James Pope, Michael Spicer, Jess Jung and Steve Oldenbourg).
About Black Hat
Black Hat is the cybersecurity business’s most established and in-depth safety occasion sequence. Based in 1997, these annual, multi-day occasions present attendees with the newest in cybersecurity analysis, growth, and developments. Pushed by the wants of the group, Black Hat occasions showcase content material immediately from the group by way of Briefings shows, Trainings programs, Summits, and extra. Because the occasion sequence the place all profession ranges and tutorial disciplines convene to collaborate, community, and focus on the cybersecurity matters that matter most to them, attendees can discover Black Hat occasions in the US, Canada, Europe, Center East and Africa, and Asia. For extra data, please go to www.blackhat.com. See the press launch for Black Hat Asia 2024.
We’d love to listen to what you assume. Ask a Query, Remark Under, and Keep Related with Cisco Safety on social!
Cisco Safety Social Channels
Share:
