Making Sense of the Sisense News

Brian Krebs has reported that "Those sources said the breach appears to have started when the attackers somehow gained access to the company’s code repository at Gitlab, and that in that repository was a token or credential that gave the bad guys access to Sisense’s Amazon S3 buckets in the cloud. Both sources said the attackers used the S3 access to copy and exfiltrate several terabytes worth of Sisense customer data, which apparently included millions of access tokens, email account passwords, and even SSL certificates."

In this blog, we explore access token exposure, looking at some real-world examples and share how affected organizations can mitigate their risk. 

Vulnerabilities In The Software Supply Chain?

A breach of any tool that is embedded in the software supply chain and has access to customers’ own API keys, risks exposing huge amounts of data, or potentially lets an attacker walk away with those API keys to use.

Any breach involving the software supply chain is particularly impactful, greatly expanding the fall-out from a single incident and sending security teams into a panic as they scramble to understand their level of risk. Software supply chains are seeing an increased focus from attackers, who see supply chains as a valuable entry route into their targets, and increased focus from customers who want to thoroughly scrutinize the security practices of their vendors. 

What Action Should I Take? 

In this case specifically, Sisense’s CISO has shared advice and instructions with customers, which Krebs also shares in his blog.

If you use Sisense, rotate any and all keys/tokens/SSL certs that Sisense would have had access to immediately. It would also be wise to check any application access logs for those keys/tokens/SSL certs and check in with any of your impacted vendors to determine if any malicious access took place. The Sisense breach may indirectly impact organizations if their vendors use Sisense as a subprocessor. It is advisable to reach out to vendors to establish the impact on data. 

In addition, to prevent and or limit any exposure-related breaches like this, our advice is to:

• Regularly check in with vendors within your supply chain to ensure that they have proper controls in place for the protection of your data, to include authorization keys, tokens, etc.
  • Evaluate controls specific to storage and access of said keys
• Use proper access control restrictions around keys that live within third parties:
  • Set only needed permissions for said keys/tokens/SSL certs that are needed to perform their function vs. given full admin rights
  • Do not reuse keys across multiple services
  • If applicable, lock keys to specific connections, i.e. IP addresses or network connections
• Regularly rotate keys both internally and those stored with vendors to ensure exposure is limited during a potential breach
• Where possible, actively monitor the use of said keys/tokens/SSL certs to ensure proper use

How Can Ethical Hackers Help Discover Access Token Exposures?

Ethical hackers are submitting reports for data exposure of API access to HackerOne customers every day. For example, the high-profile report from 2021 to Shopify by @augustozanellato found a valid GitHub Access Token that had read and write access to Shopify-owned GitHub repositories. Upon validating the report, Shopify was able to immediately revoke the token and performed an audit of access logs to confirm no unauthorized activity had occurred. 

Other examples of disclosed reports of this type include:

• A report to Uber by @tomnomnom where they found a username and certificate that allowed API access to Phabricator on code.uberinternal.com. This API access could have given away Uber’s source code and private phabricator instance.
• A report to GitLab by @vakzz about a vulnerability that allows an unauthorized user to import local Git repositories through the RepositoryPipeline feature. 
• @txt3rob found one of Snap's internal Kubernetes instances exposing an API endpoint without authorization to the public. With access to this API, he was able to run arbitrary code/jobs as a cluster-admin and gained access to credentials with internal access to a significant number of instances.

These are all examples of similar vulnerabilities that were found and reported before they were exploited or information exposed. Running a bug bounty program is the best way to catch the most obscure flaws, but security is a process, and there’s always going to be an instance where something isn’t found in time - no security method is 100% effective. However, we do think that the best way to incentivize the discovery of bugs like this is to run a public program with a broad scope, which increases the chances that an ethical hacker will actively be looking to find that fatal flaw to responsibly disclose. 

Conclusion

There are people who will look at a large-scale breach like this and curse infrastructure, APIs, and the “inherent insecurity of SaaS”, but like all things, we have to weigh risks against gains. Any breach in the software supply chain is a reminder that we should all have well-documented and clear key rotation processes, data flow diagrams, and risk assessments of our tooling. We have to weigh the resilience, speed, and lack of maintenance costs of deeply embedded SaaS tools against the risks of a breach. It’s also a reminder that we can all level up our detection and response initiatives to identify new and unique patterns from potentially exploited APIs or stolen keys.