Magic Keys – TechReckoning Dispatch v3n2

The TechReckoning Dispatch, Vol. 3, No. 2. Sunday, March 6, 2016. In this issue: The FBI and the Slippery Slope, Cloud-based Control Planes, VMware Brain Drain?, Docker Data Center, Container Orchestration, Open Source Is Eating IT, Not Your Father’s Disk Drive.
We had a big winter storm last night here in the San Francisco Bay Area. (Winter storm, for those of you not in California, means rain.) We live in a tiny coastal town with lots of trees and lots of above-ground power lines running over the mountains. Our power went on and off dozens of time throughout the evening. Luckily all the electronics survived. We made the right choice by going to the restaurant across the road for chicken pot pie and electricity — we later met refugees from our other option, the closer sushi place, where they were reduced to drinking sake in the dark and paying with cash like cavemen.

But I thought a lot about ops last night, and the folks who are on call to ensure sure our infrastructure runs smoothly. Even if you’re not out repairing live power lines in a storm, but instead just restoring backups from last week, I salute you! (And if you don’t have working backups, I both salute you and kind of self-righteously blame you a little.)

Bring Me A Master Key

Let’s talk about the FBI-Apple dispute over unlocking that iPhone. This one is both easy and tough to write about. Easy, because it’s caught national attention. Easy, because all the tech media sources I read all agree with Apple for the most part, and a large number of other organizations support Apple. (Here’s the full list of amicus briefs from Apple – they are fascinating and pretty easy to scan if you skip all the legal stuff up front.) But also frustratingly hard because the public discourse, the legal discourse, and the technical discourse are all vastly different.

Here’s a good explainer from and the full Ars Technica coverage if you haven’t been keeping up.

It’s easy to fall into slippery slope fallacies when talking policy, but it’s important to note that not all slippery slope arguments are fallacious. In this case, the FBI and NY DA have stated that they have other cases with iPhones waiting in the wings, and one assumes the many other agencies are as well. This case was chosen by the FBI for maximum emotional impact.

Forensic scientist Jonathan Zdziarski notes that what the FBI is really asking for is a forensic “instrument”, and for that to be used in court, we’re talking about much more than just a quick hack. Here is just the first paragraph of a full post about what creating and certifying an instrument entails. Apple, FBI, and the Burden of Forensic Methodology

The tool must be designed and developed under much more stringent practices that involve reproducible, predictable results, extensive error checking, documentation, adequate logging of errors, and so on. The tool must be forensically sound and not change anything on the target, or document every change that it makes / is made in the process. Full documentation must be written that explains the methods and techniques used to disable Apple’s own security features. The tool cannot simply be some throw-together to break a PIN; it must be designed in a manner in which its function can be explained, and its methodology could be reproduced by independent third parties. Since FBI is supposedly the ones to provide the PIN codes to try, Apple must also design and develop an interface / harness to communicate PINs into the tool, which means added engineering for input validation, protocol design, more logging, error handling, and so on. FBI has asked to do this wirelessly (possibly remotely), which also means transit encryption, validation, certificate revocation, and so on.

On the another axis of the slippery slope, Apple also argues that in the future they could be compelled to secretly turn on tracking functionality like location services or the microphone. The fundamental question is what can companies be compelled to build for law enforcement? Can we as a society solve this before the whole web of IoT device sensors is fully built out?  (Here’s a link to the annotated Apple filing. People who have an Amazon Echo really like it. The comparison to 1984’s telescreen has to be made at least occasionally.)

The bigger, base issue in my opinion is a disavowal of mathematics. Crypto experts tell us that no secure backdoor is possible. We have all seen the exploits of software that was designed to be secure, and exploits where backdoors were put in for other reasons, and that’s without the vendor making exploit utilities — just read Zdziarski’s post above and ask yourself if you think that software could ever escape from Apple’s lab.

Even Michael Hayden, ex-head of the CIA and NSA, agrees in this fascinating KQED radio interview that strong encryption is necessary for our society. (I certainly don’t see eye-to-eye with him on the rest of the interview, but let’s focus on the agreement.) Michael Chertoff, former Secretary of Homeland Security, at the RSA Conference last week likened Apple’s creating a cracked iOS to creating a biological weapon in a lab that someday could escape (again, see Zdziarski). Also at RSA and quoted in that article, former NSA Director Mike McConnell said “ubiquitous encryption is something the nation needs to have,” and he was even the Clipper Chip guy. (Remember Clipper Chip?) Secretary of Defense Ash Carter seemed to support strong encryption, albeit with a few waffle words. However, on the RSA Cryptographers Panel, Adi Shamir, the S of RSA, disagreed with the rest of the panel (including Ron Rivest, the R of RSA along with Moxie Marlinspike, Whitfield Diffie, and Martin Hellman), and was on the pro-FBI side. So we remain at some level divided.

And for comedy relief we have the ridiculous: San Bernardino DA says seized iPhone may hold “dormant cyber pathogen” by David Kravats at Ars Technica.

The latest volley was today’s Washington Post op-ed from Apple VP Craig Federighi. He answers the question that’s been in the back of my mind the whole time — if all your info is available to the law enforcement once it gets stored in a cloud-based service like iCloud or Gmail, then why the big fuss over the device? Anybody who has dealt with end-point security will echo what Craig says:

But the threat to our personal information is just the tip of the iceberg. Your phone is more than a personal device. In today’s mobile, networked world, it’s part of the security perimeter that protects your family and co-workers. Our nation’s vital infrastructure — such as power grids and transportation hubs — becomes more vulnerable when individual devices get hacked. Criminals and terrorists who want to infiltrate systems and disrupt sensitive networks may start their attacks through access to just one person’s smartphone.

Have an opinion? Hit Reply and share your thoughts.

Are “Cloud Control Planes” a Thing?

One of the interesting parts of Platform9 is that it is a cloud service that controls your on-premises resources. Startup Skyport Systems also has secure on-premises hardware with a cloud-based control plane. You could think of CloudPhysics the same way (on-site agent, cloud-based service). Another currently 100% cloud-based startup was telling me about similar plans to manage on-site resources. Even companies like Nutanix and VMware run centralized phone-home health services for their products. A lot of noise is being made about hybrid cloud. Are cloud-based control planes (CPaaS?) of on-premises resources A Thing?

On VMware and “Brain Drain”

There are new articles that make us actively dumber, and I’m not talking about 13 Cats That Look Like Donald Trump. This week, Eugene Kim’s article in Business Insider, There’s a serious brain drain going on at VMware that no one’s talking about is one of those articles. In it, Eugene lists 11 execs that have left VMware. First of all, he stretches back to Ramin Sayer’s departure in Dec 2014, 15 months ago, almost as if we was trying to build a case. Secondly, five of the 11 are from the troubled vCloud Air unit. No surprises there. Of the recent departures, both Martin Casado and Carl Eschenbach became top-tier VCs, a profession that involves trading in your United Global Services status for a Tesla and a dump truck of money that backs up to your garage. Also for both, I’m not sure if they had any further place to go, career-wise, inside VMware. I’m sure the Dell takeover of EMC and especially the resulting drop in stock price are pressuring retention, but the executive turnover looks pretty normal to me. A few other thoughts:

  1. I’m frankly more worried when the smart techies leave than when the execs leave. My friends who work at VMware are still engaged, having as much fun as one does in a biggish company, and excited about VMware’s future.
  2. Neither VMware’s early team nor its golden years team were 100% never-to-be-replaced olympian gold-medal winners. They had their share of dysfunction, short-sightedness, and mis-execution. Not all executives, even the good ones, are good fits for every role at every stage of a company. Don’t assume every executive change is bad for a business. Even when a well-liked, effective exec goes, room is left for new blood.
  3. Speaking of which, the current team — including Raghu Raghuram, Maurizio Carli, Sanjay Poonen, and Ray O’Farrell — are solid. My one comment both about VMware and the future Dell-EMC exec lineup is aside from Dell Chief Customer Officer Karen Quintos and VMware CMO Robin Matlock, it’s a big sausage fest.
  4. VMware, because it was well loved, is also scrutinized above other companies and subject to a wistful good-old-days attitude. Get over it.

Disclaimer: Still long $VMW

Worth A Click

Docker Datacenter and Docker Cloud have been released. Read more in The Container Orchestration Battle is Heating Up by Jonathan Baier at the Cloud Technology Partners blog.

This brings up two interesting thoughts: 1. Docker Datacenter is designed to run in your, um, data center, not the public cloud. Docker is not just about the public cloud. More details and architecture diagrams from Timothy Prickett Morgan at the Next Platform. Docker Trickles Down From Hyperscale to Enterprise.

The interesting bit about Docker Datacenter is not only that it is designed to be deployed inside of enterprise datacenters for local use on applications running behind the corporate firewall, but can also be used as a substrate to provide a compatibility layer on raw infrastructure clouds and, in a sense, compete against the container services that AWS, Google, and Microsoft have created. This kind of cross-platform compatibility was not possible at the server virtualization level, given that these companies standardized on different hypervisors in their public facing clouds (Xen, KVM, and Hyper-V, to be specific) and use a very different set of APIs and tools to control that virtual infrastructure.

And thought #2. Docker is laying out chips on the table to indicate it’s in a serious commercial game vs both open source projects and commercial competitors. See also Parity Check: What Does Container Orchestration Actually Mean? by Lawrence Hecht at The New Stack.

Unfortunately, there is little agreement about what an orchestration product does. Is it limited to service discovery, cluster management and configuration management? Are Kubernetes, Docker Swarm and Mesos comparable? Where do PaaS offerings (e.g., OpenShift, Deis, CloudFoundry) or configuration management tools like Chef, Ansible and Puppet fit in? Interpreting market share will be difficult until the industry struggles answers these questions.

Bernard Golden hits it out of the park as he sums up his current thinking about IT. 4 principles that will shape the future of IT at Note he’s not saying that everybody has to be a DevOps engineer — but that the value, and tools, of IT are indeed changing.

  1. Software is eating the world.
  2. Open source is eating the technology industry
  3. No enterprise is Netflix… or will ever be
  4. Helping enterprises becoming software companies via open source is the next decade’s IT challenge … and opportunity 

In another great paper from FAST ’16, Google introduces the concept of Disks for Data Centers. They’re not ready to go 100% solid state yet, and if you had to deal with YouTube, you wouldn’t be either. Even the section on form factor is fun.

We propose increasing the allowable height (“Z height”). Current disks have a relatively small fixed height: typically 1″ for 3.5″ disks and 15mm maximum for 2.5″ drives. Taller drives allow for more platters per disk, which adds capacity, and amortizes the costs of packaging, the printed­-circuit board, and the drive motor/actuator. Given a fixed total capacity per disk, smaller platters can yield smaller seek distances and higher RPM (due to platter stability), and thus higher IOPS, but worse GB/$. The net result is a higher GB/$ for any specific IOPS/GB that could be achieved by altering any other single aspect, such as platter sizes or RPM alone. It may also be that a mix of different platter sizes (in different disks) provides the best aggregate solution.

Keeping Up

Ever think that tech evangelists have it easy and that they all pretty much do the same thing? Well, they don’t, and evangelist Ken Hui talked with The Geek Whisperers about why he left his good job with a good company to try and find a better fit for him. Listen to How To Publicly Resign As A Tech Evangelist with Ken Hui – Episode 105.

Just Hit Reply

What dystopian future are we headed to when we lose our strong encryption? Think the FBI has ulterior motives? Or do the techies just not get it? Just hit reply on this note and send an email to and I’ll include your comments next week.

The TechReckoning Dispatch. A periodic newsletter from John Mark Troyer with links and opinions about enterprise technology. Embrace the light side of the force this week and see how that goes. True story, I made up the “cats that look like Trump” joke, then Googled it and it turns out to be a real thing. I declined to link out to any of the articles to protect you from getting dumber. ArchiveSubscribeEmailTwitterFacebookWebsite. “Each evening, from December to December, Before you drift to sleep upon your cot, Think back on all the tales that you remember Of Camelot. Ask ev’ry person if he’s heard the story, And tell it strong and clear if he has not, That once there was a fleeting wisp of glory Called Camelot. Camelot! Camelot! Now say it out with pride and joy! Camelot! Camelot! Yes, Camelot, my boy! Where once it never rained till after sundown, By eight a.m. the morning fog had flown… Don’t let it be forgot That once there was a spot For one brief shining moment that was known As Camelot.

posted by on March 7, 2016