Attribution is hard. Out of all the digital forensic disciplines, it is probably the hardest.

Digital forensics is nothing like what you see on TV – on so-called cyber-CSI shows, the investigator types in a few magical keystrokes and evidence comes flooding out of the completely unlocked computer. A few more keystrokes and a magical graphical app “backtraces” the perp all the way to his house switching on his webcam and locking his bedroom door.

The reality is far less sexy.

In the real world, attribution involves sifting through gigabytes of assorted data through hundreds, even thousands, of machines. Each one, a scene of crime in its own right. To make it even more challenging, this is often being done in an environment that is as permanent as footprints on a sandy beach just before the tide comes in. Every fragment of data could be something that tells you who was behind the crime, or it could be a red herring – something that has nothing to do with the crime, or even worse something put there to misdirect or sabotage the investigation.

Attribution is part science, part detective work and is most definitely an art form. Some folks are really good at it, while others just aren’t cut out.  What all the people who are great at it have in common, however, is patience. Attribution is slow, often frustrating, work with many false starts and lots of rabbit holes to get lost down. Sometimes attribution can be accelerated using intelligence, however, when this happens it is important not to confuse the two.

Intelligence is not evidence. Intelligence is collected to a different set of standards to evidence and with a completely different aim in mind.

• Evidence is collected in a way that meets accepted international standards and is gathered with a specific minimum volume in mind so as to meet acceptable burdens of proof – “a preponderance of evidence”, “probable cause”, “beyond a reasonable doubt”.
• Intelligence on the other hand is collected in a way that protects the intelligence, its sources and operatives or analysts from exposure or counterattack while aiming to meet the standard of “being actionable”. A lot of intelligence would at best be considered hearsay in court.

However these differences are fine – they are tools for different jobs. CSIs generally don’t have to worry about being assassinated if they get it wrong. Intelligence operatives, on the other hand, face that risk all the time. The problem comes when we mix the two, especially when you are talking to an audience that doesn’t realize the potential differences.

Intelligence that isn’t backed up by hard evidence can also lead to terrible mistakes.

It is clear that there are many folks who have an agenda when it comes to the Sony investigation. We need to make sure that these agendas don’t get in the way of carrying out a thorough and complete investigation. Likewise, we need to be really careful that those agendas don’t damage our intelligence sources, as, let’s face it, if you were really serious about protecting that intelligence, you wouldn’t hint about it in public communications.

I hope that despite the early conclusion of North Korean guilt, we keep investigating this cybercrime.  Right now the “evidence” that has been presented doesn’t give us enough information to make any real conclusions – we can’t rule out North Korean involvement at any stage of the hack, but neither can we conclude that they were behind it. Hopefully with time, more evidence will be brought to light that enables an accurate attribution of whoever carried it out.

This article published by Bruce Berkowitz in 2003 in the Washington Post covers many of my concerns – http://www.rand.org/blog/2003/02/the-big-difference-between-intelligence-and-evidence.html