Insights from Hacker Summer Camp ‘24 

I just returned from DEF CON 32, which was my 20th consecutive year visiting the world’s largest gathering of hackers in Las Vegas. Some of my motivations for attending Hacker Summer Camp (Black Hat Briefings, Security BSides Las Vegas, and DEF CON) every year are as follows:  

• Learn about new cybersecurity tools, techniques, and technologies 

• Network with likeminded people 

• Gain industry insights on market trends and competitor strategies 

• Have a bit of fun 

The following are just a few tidbits I picked up during this year’s visit.  

Closed-doors Briefing on an Advanced Persistent Threat (APT) Group 

 Well, unfortunately, this briefing was TLP:RED, which means that information was only for specific individuals and cannot be disclosed outside that gathering. But I can tell you this much: APT groups are getter a lot smarter with their malware deployment. 

Criminal and nation-state actors now regularly use execution guardrails, which are checks and balances embedded within malicious code to restrict its execution to specific environments. Using these techniques, also known as environmental keying, the malware hides its true nature if it thinks it is in a sandbox environment (such as a VM or micro-kernel) with its actions being observed by a human or endpoint protection solution. It stays in stealth mode until it checks the following: 

• System information (OS version, hardware specs (CPU, RAM, disk space), drives, and software versions 

• Network environment (IP address, domain name, MAC address, Internet connection type) 

• User info (account name, geolocation, time zone)  

• Presence of certain file paths or registry keys, anti-virus or EDR 

Only once it determines it’s really on the desired platform and being executed by the targeted user will it drop its malicious payload.  

This briefing also had a graphic reminiscent of a Gartner Magic Quadrant, except it was for APT groups – showing the Leaders, Visionaries, Niche Players, and Challengers. I found this pretty humorous but could not locate it out there on the Web. So, if anyone has seen that, please forward a copy to me. Thanks! 

 Using AI to bypass AI 

TrustedSec founder Dave Kennedy revealed how he “always gets a shell” during a penetration test (i.e., successfully gains access to a target system’s command line interface via an exploited vulnerability). Kennedy said that he will go into GitHub and find some Microsoft published PowerShell script, which will often be hundreds of lines of complex code, and then he will embed his malicious code in it. The script, looking like trusted code, will most likely bypass antivirus or endpoint protections, allowing it to execute on the local host. He described this technique in his DEF CON 18 talk and published a tool for it called Magic Unicorn, which uses a PowerShell downgrade attack to inject shellcode directly into memory. 

Lately however, machine learning has helped spot some of this anomalous code and thwart such attacks. So, now, he uses a GenAI app, such as ChatGPT, and asks it to look through his Magic Unicorn script and identify “risky” code, and then to rewrite the script in such a way that it won’t be spotted as risky – and it will faithfully do so! 

ChatGPT Side-Channel Attacks 

Speaking of AI, I heard an interesting talk at DEF CON by some researchers from Ben-Gurion University in Israel about how AI assistants such as ChatGPT leak private information (even on an encrypted channel). Through network traffic analysis, the researchers could infer with some confidence what private conversation you were having with the chatbot. (E.g., “What do I do for this rash?” Or “How do I know if my spouse is cheating?”) 

The full story was published on March 14 in Ars Technica, but here’s the gist of how they did it … 

Generative AI breaks up text processing into tokens for processing. Depending on the engine, they often use one token per word or punctuation mark. For example, “Hello, world!” might be tokenized into [“Hello”, “,”, “world”, “!”]. 

Using the popular network traffic analysis tool Wireshark to watch data being sent back and forth from the user to the chatbot, the researchers learned that each network packet being sent was equivalent to one token – meaning that they could determine the lengths of the tokens, because none of the chatbots was using “padding” (extra characters) to hide the length of the packets.  

This fact is absolutely amazing, because cryptographers have known for years that attackers will use length and frequency analysis to break encipherment. I guess AI developers forgot this, or else as the researchers surmised, they were trying to get their product to market so quickly, they failed to account for some basic cybersecurity safeguards.  

In any regard, through feeding numerous queries (known plaintexts) to the chatbot and recording the responses, the researchers were ultimately able to predict with more than 55% accuracy what conversation was taking place – even though all the data to and from the chatbot was encrypted. 

After divulging the vulnerability to all of the leading GenAI companies, the researchers found that most of them were responsive and implemented padding features to make eavesdropping more difficult. They also published a tool for doing your own GPT Keylogger analysis: https://github.com/royweiss1/GPT_Keylogger 

Exploit Prediction Scoring System (EPSS): a Tool to Aid Vulnerability Management 

EPSS is a data-driven model designed to estimate the likelihood that a software vulnerability will be exploited in the wild. It assigns a probability score between 0 and 1 (or 0 to 100%), helping organizations prioritize their vulnerability management efforts based on the potential threat.  

Why this is important: at an RSA briefing a few years back, I learned that fewer than 5% of vulnerabilities are ever exploited in the wild. The rest are theoretical, and given that there are hundreds of thousands of Common Vulnerabilities and Exposures (CVEs) out there, enterprises need to prioritize their remediation efforts to be as efficient and effective as possible.  

Cisco engineer Jerry Gamblin explained how to apply the EPSS model. There is also a very useful site called https://patchthis.app that provides a curated list of CVEs that organizations should prioritize for remediation. The list is updated hourly and includes data from several sources.  

Cybersecurity Entrepreneurship Insights – Learning from Failure 

SCYTHE CEO and DEF CON ICS Village cofounder Bryson Bort now lives in the Tampa Bay area and has become active in our local infosec community. I really enjoyed the cybersecurity founders panel with some of his peers, in which Bryson revealed what he thought were some of the greatest entrepreneurship qualities a founder needs: Passion and the ability to put up with BS (i.e., the minutiae of running a business without quitting!).  

Paraphrasing the panelists: “The business part is easy. Much more difficult are … culture, competition, and communication.” 

Some other insights:  

• In our business, always keep in mind that security is a fraction of the IT budget, which is a fraction of the Operations budget, which is how the company makes money – in other words, you’re always drawing from the smallest fraction of a company’s overall spend. 

• Loss leaders never pan out (i.e., never give away anything for free); superior approach is better qualification of your prospects. 

• Innovation is difficult because it’s always (at least) 18 months out from delivery. 

• No matter what you do, focus on your strengths, don’t get too spread out; do what you do best. 

• The “North Star” Key Process Indicator (KPI) is renewals.