Here’s my responses to the latest comments on the Sony hack, as presented by Director James Comey at the International Conference of Cyber Security in New York this morning.

“The tools in the Sony attack bore striking similarities to a cyber attack the North Koreans conducted in March of last year against South Korean banks and media outlets.”

This is something they pointed out in their first press release.  See https://marcrogers.org/2014/12/21/why-i-still-dont-think-its-likely-that-north-korea-hacked-sony/

“We put our behavior analysis unit to work looking at the statements, the writings, and the diction of the people involved claiming to be the so-called Guardians of Peace in this attack and compared it with other attacks we know the North Koreans have done. And they say, easy for us, it is the same actors.”

OK, but what other attacks? Like some of the other evidence presented, this is interesting sounding, but without any of the actual science it’s not much more than hearsay. What are the other attacks they are comparing with? What are the points of similarity? How similar are those points?

“We brought in a red team and said what else might we be missing… and we end up in the same place.”

I don’t even know what this means. Maybe they are saying they brought in a team of professional penetration testers and asked them to use their experience as “hackers”? If so, it’s interesting but again it is not really evidence.

Update: I’m told that in FBI speak “red teaming” means the FBI brought in professional critics to debate competing hypotheses. “ended up in the same place” means that despite alternate hypotheses after reviewing the evidence the red team ended up with the same conclusion – i.e. that North Korea was behind the attacks.

I would be very interested in hearing what alternate hypotheses were presented and on the basis of what evidence they were subsequently eliminated.

“I know that some serious folks have suggested we have it wrong, I would suggest, I am not suggesting, they don’t have the facts I have — they don’t see what I see, but there are a couple things I have urged the intelligence community to declassify that I am going to tell you right now. The Guardians of Peace would send emails threatening Sony employees and would post online various statements explaining their work. In nearly every case they used proxy servers to disguise where they were coming from. But several times they got sloppy.

There’s several things here. First, I don’t think anyone would disagree with him when he says he has access to information that no one else has. I would hope that the FBI has access to a lot more information than me. However, what many of us are saying is that if you are going to accuse a foreign country of an egregious crime, and have the US respond in a punitive way to that country, the evidence should be clear, of a good standard, and handled in a transparent way. If the FBI is sitting on a smoking gun then they should tell us about it because so far all they have presented is flimsy, at best circumstantial, evidence.

Let’s look at the email claim:

First, they are saying that these guys, who so were careful to route themselves through multiple public proxies in order to hide their connections, got sloppy and connected directly. It’s a rookie mistake that every hacker dreads. Many of us “hackers” even set up our systems to make this sort of slip-up impossible. So, while its definitely plausible, it feels very unlikely for professional or state-sponsored hackers in my books. Hackers who take this much care when hiding their connections have usually developed a methodology based around using these kinds of connections to hide their origin. It becomes such common practice that it’s almost a reflex. Why? Because their freedom depends on it.

However, even if we take that to one side and accept that these emails came from North Korean IP addresses, what are those addresses? If they are addresses in the North Korean IP ranges then why don’t they share them? If they are North Korean servers, then say so! What about the possibility that this attacker who has shown ability and willingness to bounce their connections all over the world is simply bouncing their messages off of North Korean infrastructure?

Finally, how do they even know these emails came from the attackers? From what I saw, the messages with actual incriminating content were dumped to pastebin and not sent via email. Perhaps there are messages with incriminating content – and by this I mean links to things only the attackers had access to – which they haven’t shared with us? Because from where I am sitting, it’s highly possible that someone other than the attacker could have joined in the fun by sending threatening messages as GOP, as we have already seen happen once in this case.

It would be far more interesting to me if these IP addresses were from connections that resulted in data being moved in or out of the network or connections involved in the control of compromised systems. For example:

• Connections that can be directly implicated in placing tools into the network or logging into Sony machines
• Connections involved in directly ex-filtrating the terabytes of information that GOP stole from Sony out of the network.

Once you have thoroughly excluded the risk of these connections being bounced through a compromised “patsy” system or through a series of proxies.  Once you are able to demonstrate that they are direct, or that you can account for every single point or hop in the connection. At this point, it would be hard for the connections to be anything but as a result of the attackers themselves.  However, the risk that somehow the connection is being redirected or that you are being misdirected is VERY hard to eliminate.

This is what makes attribution is so hard.

“Several times, either because they forgot or they had a technical problem, they connected directly and we could see them. And we could see that the IP addresses that were being used to post and to send the emails were coming from IPs that were exclusively used by the North Koreans.”

I think I covered this above.  I agree that it is certainly possible for an attacker to make mistakes like this, but it seems very unlikely. It would be a rookie mistake for an accomplished hacker. However, people do make mistakes.

Equally, however, it could be a sign that different people are involved – one who is being extremely careful, and one possibly an opportunist who is sending lots of threatening messages. Perhaps the hacker and an opportunistic emailer taking advantage of the chaos?

Finally, what are the IP addresses, and how can you say “exclusively North Korean”? The only addresses that I can think of as exclusively North Korean are ones that belong to the North Korean IP ranges, and if that’s the case, why not say so? Those addressees are hardly a matter of national security. Also, how did they rule out the possibility that someone is faking those IP addresses, or bouncing off of them, or hijacking them?

“It was a mistake by them that we haven’t told you about before that was a very clear indication of who was doing this. They would shut it off very quickly once they realized the mistake, but not before we saw them and knew where it was coming from.”

“We have a range of other sources and methods that I will continue to protect because we think they are critical to the entire intelligence community’s ability to see future attacks and understand this attack better.”

So we are back to the SIGINT. I respect their need to keep matters of national security discreet. After all, disclosure can jeopardize the operation, the source, and any future use.  However, if SIGINT is the only really strong evidence they have then say so. Stop presenting the rest of this weak circumstantial evidence, passing it off as a smoking gun.

“I remain where I started not just with high confidence but VERY high confidence that the North Koreans perpetrated this attack.”

Likewise, while I agree there is some interesting information here, I see nothing that conclusively incriminates the North Koreans. Instead, if anything, this evidence just leaves me with even more questions.

“We are still looking to identify the vector – so how did they get into Sony. We see so far spear phishing coming into Sony as late as September this year. We’re still working on that, and when we figure that out we’ll do our best to give you the details on that. But that seems the likely vector of entry into Sony.”

Wow. That’s potentially a HUGE bombshell. Is the FBI really saying that they don’t know what the vector was, or are they just being coy? If they genuinely don’t know what the vector was then I have even more concerns. The vector is often the single most important piece of evidence you can uncover in a cyberattack like this.

I don’t mean to be blasé because really do I appreciate that running an investigation like this is a very difficult exercise with many moving pieces. However, for me, the vector is a HUGE factor in understanding what happened and ultimately in correctly attributing the criminals that perpetrated it.

I do also appreciate that it is easy commenting from the peanut gallery, and not so easy to be sitting in the hot seat, especially with the kinds of constraints that the FBI have.

Finally a few folks have asked me what I would like to see in order to be convinced that it really was North Korea.

That’s a tough question to answer. I would love to see a smoking gun but a smoking gun can mean so many things in the world of digital forensics. It’s also something that you so rarely get. Some of the things that would help, though, would be –

• Evidence of direct symmetrical connections – i.e. either data connections or control connections that were not routed from a proxy. Not asymmetric connections such as an email moving from server to server.
• Connections that were routed through proxies supported by logs from every single proxy involved.
• Comprehensive forensic evidence from a “staging system” used by the attackers to control or deploy their attacks.
• Evidence of highly specific code, which can then be tied to an individual or individuals.
• Evidence of a highly specific vulnerability or 0day that can be traced.
• Details (not just hints) about highly specific tools used by the attackers, which can be linked back to the attackers.
• The attacker’s laptop with tools, code and logs intact.
• A confession 😉