Follow-up to my Open Letter.

Can you believe it has been more than four weeks since DEF CON wrapped up? Im just shaking the ConCrud and still wake up every night at 4am wanting a status update. Anyway, I wanted to follow up, clarify a few things, and let you know what the next steps will be.

This blog post will focus entirely on the DEF CON side of things. A blog in the not too distant future will look at tackling the much bigger problem of these security measures going national. I don’t know about you, but I don’t want to see our hotels become like our airports.

Over the last couple of weeks I have received thousands of messages of support. So many that my phone has had multiple fits. The overwhelming message was “You don’t have our permission to quit, we want to support you in fixing this.”. Got it.

Thanks for the vote of confidence. It’s humbling in many ways to know you are behind me. I received messages from hackers from all over the world, even those I consider my personal heroes.

So, I have much work to do.

By now you will have read on the DEF CON site about our meeting with Caesars on the DEF CON site. It was generally positive but I consider this the first, baby, step on a very long road.

First they clarified that the detailed policy statement released during closing ceremonies ones was not given to either me or DT before the event. It became clear that it was received by DEF CON just before closing ceremonies at 4:11pm. Not “months in advance” as some believed. Prior to this, DT assured me that other DEF CCON staff received little more than the info already made public in the Las Vegas Review Journal in February of this year:

Now, lets unpack that policy statement and how it applies to what happened:

1. Setting “do not disturb” or “room occupied” hanger on your door for “too long” triggers a security visit. What the exact parameters are remain unclear. Registration told me it was three days but I know of rooms inspected after much shorter times.

2. A security visit is supposed to be carried out by two hotel security staff. Staff are supposed to be clearly identifiable as such by both uniform and ID tag. They are supposed to announce themselves to any possible occupants and provide ID when requested. They are supposed to be using their own access cards, not ones from housekeeping or other departments.

3. All hotel security staff can be validated by calling Ceasar’s security dispatch. The dispatcher should be able to confirm the name, description and ID for the officer. They should also be able to confirm that this is a genuine security check.

4. A security visit should ONLY constitute a visual inspection of the room. Guest’s belongings should NOT be interfered with, drawers and cases should not be opened etc

5. If an officer notices a “red flag” then they are supposed to place a radio call to dispatch for instructions.

One last point from my own experience of working with Caesars over the last few years, their security team security handles incidents In very specific ways and have done for a long time.

For example, If an item of contraband is found (as in the case involving a pair of lock-picks) then they have a very specific protocol:

  • They have been using this protocol for years as it is as much to protect their officers as it is to maintain chain of custody on the item.
    • they secure and lock the room,
    • they attempt to locate and inform the guest
    • when the guest returns they identify the item visually with security staff and the guest present.
    • The item is then secured in the presence of the guest and held by security, or the matter is escalated per local laws. This means they should not “just take” any items from your room.

So what are some of the things we learn from all this?

1. Missing items with no explanation should be immediately reported as stolen.

If your soldering iron, medicine or gadgets vanished, they were likely taken without any authority and security is the last group that comes to mind. Hotels have had room theft problems for years.

The complete randomness of the items reported suggest to me theft more than policy based confiscation. Hotels have a diverse burglary ecosystem, from people using stolen keycards, to corrupt housekeepers. They even street gangs called “knockers” who walk through the hotel testing doors to see if they have been closed properly.

Pro tip: Knowledge of when something went missing is critical. All Vegas hotels have CCTV and with a time window they can likely identify the theft and make a recording of the thief. I have personally helped hotels track down thieves using accurate theft reports and good CCTV. I expect no less from Caesars

You should report any missing items as stolen NOW.

LVMPD has an online link that allows you to report stolen property:

Wile it probably won’t get your stolen item back, you will need it for insurance purposes, and more importantly getting as many of these filed as possible will give law enforcement a handle on the size of the problem. This is something they need to fix before something terrible happens to one of their guests, drawing attention to it is one of the best ways.

2. The officers who messed with property, opened drawers, opened suitcases or fiddled with items were clearly in breach of their own policy as stated. We will work out what this means as we continue our discussions with Caesars.

All of this this is something I personally hate as a security practitioner:

  • The badly implemented and apparently poorly communicated policy.
  • The huge variance in reports on what security guards did or looked for showing an apparent lack of consistent strategy or training.
  • The ineffectiveness of their approach.

Just like many others in the Infosec industry, I have had some specific training in this area. If you want to find observable flags that indicating threats of any kind, you need tried, tested, consistent techniques and consistent training.

I’ve heard of rooms where spent brass from shooting ranges (empty bullet casings) were left on the table but instead the guards walk around, photographed nightstands, luggage and personal papers. That implies to me they were either not properly trained in what to look for or didn’t really care.

3. Officers who failed to identify properly, or who showed damaged or illegible ID are also in breach of this policy as stated. This is a disaster waiting to happen. Most of us could duplicate those ID cards with a few minutes access to a photocopier. This is a disaster waiting to happen. How longer before a bogus security guard with dubious ID demands access to a lone guest’s room for criminal purposes? By doing this, hotels are training their guests to be victims.

4. Officers who “just walked in” without announcing themselves are also in breach of policy and even more concerning. I know of at least one lone female engineer who was disturbed while getting dressed and another guest who was interrupted while bathing. Its hard to wrap my head around how scared they must have been. These situations could have been far far worse. They cannot be allowed to happen again.

While most of the reports came from attendees staying at Caesars, it’s clear this was much bigger than just one hotel. I got reports over twitter from dozens of Las Vegas hotels. The most interesting thing was just how much variation their was in how this was applied. Some hotels phoned up, used it as an excuse to freshen the room and left chocolate. While others practically kicked the door down and acted like they were executing a warrant. Las Vegas has MUCH work to do if it wants to continue to be seen as a safe destination for hotel guests.

Also, despite claims to the contrary, there were definitely searches at Mandalay Bay. At least one of these was reported on in detail (along with several other incidents) by Seth Rosenblatt from The Parallax:

“A security researcher and Black Hat attendee who requested anonymity because of the sensitive nature of this story told The Parallax that a Mandalay Bay security employee threatened to bring “more people” to his hotel room door if he didn’t open it.”

Finally, while K9’s units were also spotted at the Venetian and Wynn hotels, a quick search on the internet shows that these units have been in place for several years. The fact that ALL the K9 units disappeared from Caesars the moment the last DEF CON guest checked out just strongly implies to me that that was security theatre and not any serious attempt to keep guests safe.

If we are going to keep doing something like this (and thats a big if), as a minimum Las Vegas hotels need to get together and get on the same page. They need to build a transparent, auditable policy with full accountability for those carrying it out. Guards performing this duty must be clearly marked with industry accepted quality ID. It should also be possible to see who inspected your property and when. Lastly there need to be clear lines of escalation so that guests who have lost items or feel violated know who to call and what to expect.

The alternative is likely to be a series of class action law suits with the final outcome being Hotels potentially losing the flexibility they have under the 4th amendment.

5. It seems to me that thieves took advantage of the confusion and used it burglarize rooms with cloned staff key cards. I myself lost several thousand dollars in phones.

In most cases it was small valuable items, prescription medicines and cash i.e. it looks like it was tweakers. So please file a report with LVMPD thats the best way to get this on everyone’s radars

Missed opportunity?

Speaking as a hacker, this also feels like a HUGE missed opportunity for Vegas. No city knows more about the identity of its guests and their behaviours than Vegas. Most hotels know a huge amount about you before you even check into the hotel itself. Their entire economy is built on it. Why not use all that information?

Other countries and facilities have solved this type of issue much more effectively by combining data, people, processes and technology. We already live in a world where big data can predict your shopping habits down to what illness you have or whether you are shopping for a new baby – it seems like much more of this could be solved in an intelligent, data driven way, instead of the incredibly invasive yet ineffective manual processes we face now. Many of these kind of vetted ID systems are already in place in vegas for other purposes. Adapting them could be an excellent way forward.

In my opinion everything we experienced was largely security theatre. Even the Caesars K9 units were all gone by the time the last DEF CON attendee had checked out. I do not feel it would do much to protect guests from active threats such as the one faced last October. If these hotels really want to protect their guests, then they couldn’t do better than engage the hacker community in a positive way rather than an adversarial way.

Over the next few months we will be working to get a transparent, codified policy so that we know exactly where we stand.

Personally I will be working with the Hacker community, EFF and ACLU to look at better ways to challenge these issues, and ultimately come up with better ways address the risks raised by October’s attack. We need smart, effective ways to keep our people safe or in the long run the terrorists win anyway.

Meanwhile, lets see how Vegas handles the next shot show…… who wants to bet this all goes away just time time for it?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s