Thursday 21st & Friday 22nd June 2018
The Clayton Hotel, Cork City

Randy Shoup
Moving fast at scale

Most companies slow down as they get larger, but some actually get faster. This talk will discuss the speaker’s experiences leading high-performing engineering teams at Google, eBay, and Stitch Fix, and will discuss the organization, the processes, and the culture that can help a company move fast – and even accelerate – as it grows.

Modern software-service models take advantage of the great benefits in having the same team both build the software as well as operate it in production – we call this DevOps, or simply “You Build It; You Run It”. What does this mean in practice?

Organizationally, it means small teams with well-defined areas of responsibility, directly aligned with the business. The teams are cross-functional, meaning that each team has all the skill sets it requires to do its job, while at the same time relying on other teams for supporting services, tools, and libraries.

Process-wise, it means doubling down on practices like test-driven development and continuous delivery. Using continuous delivery practices, high-performing teams can and do release their applications and services multiple times a day. This enables them to iterate rapidly, experiment courageously, and fail more quickly.

Culturally, it means end-to-end ownership. Each team owns its software end-to-end, from design to development to deployment to retirement. The same engineers who are responsible for the features are responsible for quality, performance, operations, and maintenance. This ownership puts incentives in the right place to encourage building maintainable, observable, and operable systems from the start.

All these techniques and approaches are available to everyone, and practical examples in this talk will help other organizations on their journey.

Randy Shoup

VP Engineering at WeWork

Randy Shoup is a 25-year veteran of Silicon Valley, and has worked as a senior technology leader and executive at companies ranging from small startups to mid-sized places to eBay and Google. Randy is currently VP Engineering at Stitch Fix in San Francisco.

Earlier, Randy was Chief Engineer at eBay for 6 12 years, where he was responsible for multiple generations of eBay’s realtime search infrastructure. He was CTO and co-founder of a startup, and learned just how difficult and different it is to build a company from scratch. He was Director of Engineering at Google for Google App Engine, building and operating the world’s largest platform-as-a-service. He also spent a year and a half applying eBay and Google lessons consulting with startups and large enterprises on how to improve their organizations and technology.

He is particularly passionate about the nexus of culture, technology, and organization.

Can you tell us a little bit about yourself and what you are working on right now?

I just joined WeWork about two weeks ago. So very new to my new job as a Vice President of Engineering there, where I run one of our four business areas. My team is working on how we make the experience better for new members of WeWork. We’re also looking in the future to instrument spaces with IoT devices so we have a better idea of utilization and how we can help people utilize their space better.

For the previous 2 years, I was VP of Engineering at StitchFix, which is a clothing retailer in the United-States, if you’re not an american we’re not yet in your country but it’s a pretty cool way to buy clothes: you fill out a really detailed profile about yourself and instead of shopping on our site, we send you a box in the mail with things we think you’re going to enjoy. We use a ton of data science and machine learning to do that, plus a bunch of human curation so it’s sort of: how can we combine together what machines are best at doing, with what humans are best at doing.

Earlier in my career, I was Chief Engineer at eBay and I was responsible for the search engine for the most part, so six and a half years trying to figure out how to do real time search at large scale, learning a lot about building organizations and large scale technology. Then I spent some time at Google running Engineering for Google App Engine which is Google’s Platform-as-a-Service, where I got some good Cloud infrastructure work.

I’m an Engineer at heart so I really like the technology part of it but, as much as I love that, the really important stuff is culture and organization - because it’s what actually makes the biggest difference.

You’re a significant influencer in the DevOps movement - “The DevOps Handbook” (Jez Humble, Gene Kim, John Willis & Patrick Debois) doesn’t shy away from referencing and quoting you a lot! How did you see this space evolve over time?

First of, I guess I’m honored to be in the same space as Jez Humble and Gene Kim, because I don’t think I’m at the same caliber! They are great friends of mine, I really respect them and they have done amazing things to move our industry forward.

DevOps is a name we have retrospectively put on things people have been doing for a while. It’s not a lot of organizations, but some of them have been doing what we now call DevOps for 20 years! But still right now, the vast majority of organizations have never even heard the principles or certainly never applied them. As with William Gibson’s quote: “The future is here, but it’s not evenly distributed”.

One of the things that I am personally passionate about is taking all these things that high performing organizations have been doing for a long time, the Googles-Amazons-Netflixes of this world, and apply them to all these places that are not yet Google, Amazon or Netflix!

In terms of evolution, one obvious thing is now we can use the word DevOps and it has some meaning. There’s still a bit of controversy and no universal understanding, particularly from software vendors who would like it to mean “oh it’s using my tool”. A much better way of thinking about it is “what’s the way that we can deliver value most efficiently to our customers”. Ultimately, that’s what it’s all about, that’s what DevOps is about, that’s what Agile practices are about … and that’s what we’ve been trying to do for the whole thirty years that i’ve been in the industry!

What’s cool is that now those good practices are more widely known, but even more importantly and, this is quite a bit due to Gene Kim, Jez Humble and Nicole Forsgren’s work, we have science behind these ideas. We’ve all had a sort of intuitive understanding that organizations made up of small teams are more effective, and organizations that controlled their own destiny with full-stack are more effective… but now we have science behind it! Nicole is really the scientist of DevOps!

It still is amazing to me that the organizations that are able to move faster are also able to be better: we do not need to make this choice between “should i have speed or stability?”. That’s a classic idea like, “oh, you could move fast and break things or you can be nice and slow and careful”… it turns out that that is a false choice. The organizations that are able to move fast are also the ones that are able to have a shorter time to recovery, lower change failure rates, etc. There’s a huge difference between the high and low performers: low performers are slower and more terrible and the high performers are faster and better.

What is your number one advice for organizations starting this journey today?

First step is to read and absorb the ideas: Lean Enterprise, DevOps Handbook, Accelerate… It’s a great way of discovering things that other people have already known. As a leader trying to put those things into practice, remember that it’s not about the technology but way more about organizational culture. The step one between organization, culture and technology is strangely … organization!

Forming an organization out of small teams with well defined areas of responsibility that are reasonably autonomous from each other is the first physical manifestation of these ideas. A lot of this flows from Conway’s Law, which essentially says that you ship your organization chart, or more precisely “your organization determines your architecture”. So if you ultimately want to have a new architecture that’s built out of small modular pieces that are componentized and composable then perversely you start with an organization that looks like how you want your technology to be! The other part is that small teams will move faster, because they’re more isolated from each other, because they’re more autonomous - they’re able to iterate faster and have shorter feedback loops and that’s one of the other tenets of continuous delivery.

It is so tempting to start with the technology like “oh let’s adopt Kubernetes”, “let’s start with this continuous delivery tool” (nothing wrong with any of these ideas, by the way - I like all these technologies!). Those are all things that are good - and you should have them - but in a broken monolithic organization, introducing those tools doesn’t make much difference… and again perversely, if you are in an organization made up of small autonomous teams who have good areas of responsibility, it almost doesn’t matter what the technologies are… OK, it kind of does, but only really as a second order optimization!

The hard thing is to get the organization and the culture in place, and I mean this as an engineer who loves all the technology. It’s fundamentally an organizational and cultural problem that technology can improve; it’s not just a technical problem.

You’re running a 1-day workshop at RebelCon called “Scaling Technology and Organizations Together”, could you go through what you will cover?

It will cover all the things I was just talking about, expressed more concretely in the workshop. It’s a full day on how to scale organizations and technology together, and the idea is that you first start with organizational and cultural practices: you form your organization out of small teams, with a culture of continuous learning, of not blaming people, of celebrating things that you learn. The next part is about the technical and architectural implications. The equivalent of small teams with well defined areas of responsibility are individual services and applications that are small and clean and composable, maybe connected by events.

The idea of this workshop is to express this “systems thinking” both in an organizational form and in an architectural form. Another way to think about it is: what is the training course I wish I had before I became a CTO? I learned this the hard way, through all the scars. I have given this workshop a number of times over the years and I hope it will continue to help other people not make the same mistakes I did - you can make your own mistakes!

It’s definitely geared at Engineering Leaders. These are not technical problems, but mostly organizational and cultural problems that technology can help on. The audience is people who are new to the Engineering leadership and also for people that have been in Engineering leadership for a while.

What makes Google, Amazon, Netflix fast even though they’re big? This workshop is about finding the answer!

What do you hope the participants to be able to bring back to their daily job from your workshop?

I think the most important thing is to bring back maybe a new attitude and having a new set of tools in your tool box - organizationally, culturally and technically. A lot of people that have attended this workshop for the last several years are themselves already sold on these ideas, but need a little help with putting them into practice.

More importantly, they need to convince the rest of the organization that this is the way to do it! What people left with in the past is more tools in their toolbox and also more ammunition: these aren’t just cool ideas because I want to work in this nice new culture, they actually work, there is science behind them, and here are the other examples of companies that we admire that practice them.

You also have a presentation on the conference day, called “Moving Fast at Scale”, could you tell us a bit about it?

“Moving fast at scale” is more a keynote-length exploration of the same ideas. The workshop is much deeper and broader and more participatory - that’s the other angle: that people sharing their own experience and ideas is very valuable for other attendees. The keynote is a bunch of the same ideas but more compressed, more “tweet size” if you will!

Sponsors

Follow RebelCon