Cloud Security Alliance Webinar Recap: Avoid the Breach with Shopify’s Andrew Dunbar

In this Cloud Security Alliance (CSA) fireside chat, Shopify’s VP of Security Engineering and IT sat down with HackerOne’s Luke Tucker to talk about how the e-commerce leader approaches security, from the strategic to the tactical.

How Shopify develops and improves their application security strategy

Topic 1: How do you tackle application security at Shopify?

As the company has grown, automation and high-leverage tooling have been key to their appsec strategy.

To keep up with the scale and speed of the development teams, Andrew and his security team hold strong opinions about tools and platforms that engineers use. For example, Security mandates the use of one version control tool and one operating environment. This consistency is foundational to supporting automated and repeatable techniques that allow security engineers to partner efficiently with product engineers.

Similarly, the Shopify security team is very opinionated about bringing in any new programming languages. The gating of new languages and frameworks is done in tight collaboration with product engineers in order to maintain a consistent baseline.

This kind of partnership with product developers requires strong feedback loops. The security team needs good visibility to know when something isn’t working, such as an IDS. For instance, Andrew’s team leverages the system and application monitoring tooling that the product engineers stood up. With a few developers’ time, they built a great vulnerability management suite on top of it.

In order to vet a new approach - be it a tool or language - Shopify will often use their Hack Days to test it out. This was the case when they started looking into Google Cloud Platform and Kubernetes. The Hack Days provide an opportunity to build POCs and get everyone on the same page to see how something works, and what security considerations there may be.

Topic 2: Why isn’t it enough to just have strong internal security controls—phishing, identity access management, etc.?

Every modern app today depends on lots of integrations and open source packages. From a security standpoint, this means you’re only as strong as your weakest vendor.

Therefore, you need to be super selective in deciding which third parties to work with and how much data to share with them.

Maintaining continuous visibility of the operating environment is also key to ensure your products are trustworthy. A large number of third-party integrations makes this especially important.

Having an appsec team looking at the product is also vital to maintaining customer trust in the product.

How Shopify scales security for high-growth with a DevOps methodology

Topic 3: As head of security at Shopify, how has your approach to security changed as you’ve scaled your team?

Andrew shared that when he joined the company 7 years ago, there were about 100 people, and he was the only security person.

Despite comprising the entire security department at the time, Andrew benefited from the great security culture. As an e-commerce platform that processed credit cards, there was an awareness of the paramount importance of security to earning customer trust.

As the company’s growth took off, it became clear that Andrew and his growing team had to establish automation and guardrails -- things like playbooks to ensure there was only 1 policy for incident response.

A big part of scaling is growing the security team, which posed questions like “when do we need to hire a specialist for infrastructure security and application security”.

Andrew then took the audience through a great “start from the end” exercise. He continued “the best outcome, regardless of the size of the company or the security team, is to stop a breach before it gets introduced and do this without introducing false positives and lots of noise. To do this, I needed to build a highly technical team that can engage directly with the product developers in GitHub, in a PR, and course correct away from an insecure practice or package - this requires expertise and high trust.”

Keeping with the concept of trust, Andrew introduced the concept of “Trust batteries”. He encourages his team to think about security and trust from the point of view of the many different stakeholders Shopify works with -- merchants, merchant’s customers, partners, internal stakeholders, etc.

Andrew looped back to the concept of building a great team. Yes, it means seeking out people with diverse skills and perspectives. It also means hiring for fit with the culture. 

Going further, Andrew shared that his team does not believe security is or should be perceived as, “the people with the big sticks”. “Instead, we are enablers for the business to achieve great outcomes. That means it’s about partnerships, not blame. Fit with this mindset and approach is a got to have for new security hires. People can learn the technical skills and we’re willing to invest in them to get them where they need to be.” 

(BTW, Andrew is hiring! Visit their site to check out open positions.)

Topic 4: Can you tell us a bit about how you assess and manage risks currently (specifically with your cloud-based approach)?

At Shopify, risk management starts with a great understanding of the asset in question. In terms of the risks themselves, Andrew and team strive for a consistent and simple way to assess the potential exposure of every risk.

To keep it simple, they have only two risk ratings - high and low. “If something is high, we know it needs to be fixed right away. If it is low, it can be added to the backlog and fixed later.” This is a practical setup they arrived at when they found it was really hard for a developer to know what the right action was to take on a Medium risk. “Medium doesn’t really mean anything.”

To have confidence that they are assigning the right severity to each risk, Shopify’s security team has multiple inputs into the evaluation that come from multiple perspectives. These include vulnerability management, external audits, pen testing, code review, security game days, bug bounty programs, and more. Any of these can trigger incident response and risk evaluation. Whenever a high exposure risk comes up, after it’s resolved they perform a root cause post mortem to learn how to avoid a similar situation - and ideally a whole category of similar situations - from happening in the future.

How Shopify discovers and manages critical vulnerabilities effectively 

Topic 5: What tools or programs do you deploy that have been game-changers for you?

Building applications that leverage multiple dependencies means that in order to move fast, you need good tools for seeing what versions of different dependencies you’re running. It also means you need ways to interact with the development team -- in their workflow -- so it’s easy for them to patch anything that has a vulnerability. Building out our tools so the security team can quickly see if we’re running an insecure package and so developers can easily merge the patched version is key.

Bug bounty has been huge as well - it’s a high leverage way to get high-quality reports. Knowing that a small army of experts are continuously looking at the platform and checking for security gives us a ton of confidence that our key stakeholders can trust the security of our products.

Topic 6: Having been with HackerOne Bounty since 2015,  how do bug bounties act as a critical part of your application security testing strategy?

As a web application, you need to deal with the fact that if there are problems, people will find them. So, Shopify began in 2012 with a VDP to give ethical hackers the channel to report anything they found. They started paying for bugs with HackerOne Bounty in 2015.

“This makes all our products better because it makes sure our teams know how to reply to obscure vulnerabilities that you’re not likely to find through traditional pen tests. And we also post most resolved reports publicly. Because hackers can refer to these, it has helped produce even richer reports going forward and aided hackers to look for similar vulnerabilities. All of which contributes to a very healthy culture of accountability and continuous improvement.”

You can give this lively, 30-minute webinar recording a listen here.