Skip to main content

Expanding Reputation: Introducing Signal and Impact

  • December 18th , 2015

Edited on 4/8/2016 to reflect the latest Signal and Impact implementations.

We first introduced Reputation in October 2014 to provide additional recognition to the best researchers, and to highlight quality reports to security teams. Each researcher has a Reputation based on their track record on HackerOne. Reputation has been continuously improved since its introduction and has become integral to the platform.

To build upon its usefulness as a ranking system, we launched two new dimensions to better show how each researcher's Reputation has been achieved: Signal and Impact.

Why We're Improving Reputation

The existing Reputation system is effective at measuring a consistent track record of valid results, serving as an accurate ranking function. However, it occasionally obscured quality over quantity. The introduction of Signal and Impact helps further highlight quality performance.

New Dimensions: Signal and Impact

Signal & Impact

Signal is the average Reputation per report.
Reputation is gained or lost each time a report is closed, making Signal an aggregate representation of report validity.

Impact is the average Reputation per bounty.
Reputation is gained based on the relative size of the awarded bounty, making Impact an aggregate representation of report severity.

  • Signal and Impact are only calculated when there are more than 3 closed reports or 3 bounties, respectively.
  • Signal is measured on a scale from -10 to 7, corresponding to Reputation changes for triage states ranging from "Spam" to "Resolved." Self closed reports and Duplicate reports are excluded.
  • Impact is measured on a scale from 0 to 50, corresponding to Reputation gains calculated by bounty levels awarded.
  • Impact is only calculated for reports submitted to programs that offer bounties. We also display the percentile rank for Signal and Impact, relative to other researchers on the platform.

Examples

Alice and Bob have the exact same amount of Reputation (241). But Alice has a higher Signal (4.12 > 1.35), indicating a better track record of submitting valid reports. Alice also has a higher Impact (15.83 > 9.50) and therefore, a higher chance of submitting an important report.

This is valuable information for both of them. Alice will see that she is doing quite well with her current trajectory, while Bob will see that others are more accurate about their reports, and can adjust his reporting processes accordingly.

Conclusions

For a response team, Signal provides a means of identifying researchers who have had consistently valid reports while Impact highlights those with the greatest severity. For researchers, we hope both Signal and Impact will help you benefit through an improved understanding of your performance relative to your peers. In the coming months, we'll be incorporating Signal and Impact into other visible areas of HackerOne, such as the invitation system. We hope you'll find them useful and as always, we'd love to hear your thoughts.

Recent articles

Zero Daily Newsletter: Fun, yet informative, AppSec, bug bounty, and hacker news

Read the news every day, and check the usual websites? Want to get your industry news and have a little humor…

More Hardware, More Problems

Bounties are for hardware, too. Microwaves notwithstanding, there is an increasing amount of connected…

Bug fixes just got a little easier; HackerOne introduces bi-directional JIRA integration

It’s now possible to view updates on JIRA issues right inside your HackerOne Reports. The two-way integration…