LINE Security Bug Bounty Program Report 2019

This guest blog post was authored by Robin Lunde from the LINE Security team and originally published on the LINE company blog.

Hello! I am Robin Lunde from the LINE Security team. 2019 was a very eventful year for the LINE Bug Bounty program. I would like to share with you some of the highlights and key takeaways from 2019, as well as our plans for 2020.

Transition to HackerOne

In late 2018 through early 2019, we began investigating the potential benefits of moving our program to HackerOne. After careful consideration, we concluded that we could likely improve the program and its results by moving to the HackerOne platform. As such, we started the process of migrating our program. One of the factors that made us decide to move was the option to easily make reports public, allowing us to be more transparent and open about the reports we receive. We have previously shared information under special circumstances, but going forward we want to share information on a more regular basis. Another important factor was that it would likely increase our participation globally, because our program would be easier to find by being on a more widely known platform. Also, most hackers already have accounts on the platform, and they would be able to make reports more easily without having to sign up for a separate account. 

After some time reviewing our options, we came to the conclusion that starting as a private program and running it in parallel with our own program, would let us evaluate how to best proceed. We made the necessary preparations and on July 1st, we launched our HackerOne private program. A private program is a program where only invited hackers can participate. We invited all participants from our old program, and also got help from the HackerOne team to introduce new, highly skilled researchers.
 

HackerOne Private Program
As mentioned, we opened our private program on July 1st 2019. Our first valid report was submitted on July 12th and earned a reward of $5,000. During the operation of our private program, we received a total of 101 reports, with 37 of them qualifying for rewards. This means that we rewarded over 36% of the reports we received, which is quite impressive. After seeing such good results, getting used to the new platform, and having some time to try out and adjust to the new procedures, we began the process of becoming a public program.

HackerOne Private program statistics. November only includes data up until Nov. 15th

While the difference between a private and public program may not seem like much, the initial wave of reports after becoming public is a significant change that needs to be accounted for. In addition, in case of multiple critical reports or other unforeseen circumstances, there needs to be procedures and policies in place to make sure that the team member in charge knows what to do and how to escalate the situation properly. Finally, it was important for us to spread awareness and try to encourage as many hackers as possible to participate in our program. That meant a big effort in terms of preparing press releases, sharing information with our reporters and making sure that hackers knew what was happening. After finishing the last preparations in the first few weeks of November, we became a public program on November 15th.

Going public
After four and a half months of preparation and transitioning, we finally moved our program completely to HackerOne and became a public program on November 15th. We wanted to be able to share this information during our Developer Day event, so a lot of effort went in to meeting that deadline. We received two more reports in our first week alone, than throughout the operation of our private program, totaling 103 reports from November 15th until November 23rd. While these had a significantly lower rate of rewarded reports, it confirmed that our effort in spreading awareness and information had been a success. But as they say, when one chapter opens, another one closes.

With our change to become a public program on HackerOne, there were no longer enough resources available for us to keep our old platform running. It was with a sad heart that we said farewell to where our bug bounty journey had begun. At midnight on November 14th, the last chance to submit a report through our old portal passed and we successfully completed our goal of moving to HackerOne.

Retiring Our Original Platform
We officially closed our own platform for submissions on November 14th. However, in order for our reporters to be able to read their old reports, as well as take notes, export the information, or otherwise finalize any withstanding issues, the platform was still accessible to registered users until December 31st, 2019. With the coming of the new year, we had successfully closed all issues on our old platform, and it was completely shut down. Even though our old platform is no longer operating, we wanted to make sure that the history from our early beginnings and our gratitude to those who helped us build our current program was not forgotten. We have elected to preserve our original web page, as well as the Hall of Fame from previous years, in recognition of those who have contributed to keeping LINE users safe. While the page has been updated to reflect that we have now moved to HackerOne, we intend to keep it up for the foreseeable future, both as a source of information and as a reference to our previous program.

We would like to thank everyone who participated through our old platform, we hope you will stick with us and follow us over to HackerOne where we can continue to improve and work together.

Numbers in Review

First let us review the year as a whole. Then we will split up the statistics and see how the choices we made this year influenced our program.

Whole Year In Review
From the following graph, we can see a clear indication that the number of reports received has been steadily climbing. In 2019 we received a total of 520 reports. In addition, our number of valid reports and reporters who received a reward has also steadily increased, with respectively 106 valid reports received in 2019 and a total of 47 unique hackers earning a reward.
 

If we look at these statistics in terms of growth, the following graph show the relative increase from 2018 to 2019. For example, the number of rewarded hackers increased by 42% from 2018 to 2019, meaning 2019 had a total of 142% of the rewarded hackers that 2018 had. In addition, we also saw a 20% rise in valid reports from 2018. This is quite substantial and a very important metric for us, because it means that the hackers participating in our program are providing improved reports and results. It also means we are spending less time dealing with invalid reports, compared to previous years. This is essential in order for us to make sure that we spend our time efficiently.

We also saw that we had a steady flow of reports throughout the year, with certain peaks showing clearly. It is also apparent that we received most of our valid reports later in the year. The following graph show valid reports per month, grouped by each quarter.
 

Now, let me share the numbers that you are probably most interested in - the bounties themselves. Last year, we awarded a total of $139,000 in bounties, resulting in a total of $347,000 throughout our bug bounty program. (For details regarding previous years, see the Reward statistic section found in the LINE Security Bug Bounty Program Report from 2018)

Private Program
Knowing the above, it is time to analyze our decisions and see how becoming a private program on HackerOne affected us. As we can see from the graph above, Q1 and Q2 had on average 3.5 valid reports per month. In Q3, coinciding with our private program launch, we can see that the activity increased a lot. This applies to both programs though, as they were ran in parallel at the time. What is not shown in the previous graphs though, is the high rate of valid reports received through HackerOne. For the 4 months we operated as a private program on HackerOne, we had a valid submission rate of 40.94％! In other words, almost every other report was valid. Next, let us investigate the transition to a public program.

Public Program
After going public, we saw an increase in total valid reports, but a decrease in valid report rate. This is according to what we expected beforehand, due to no longer being restricted to only expert researchers, but being open to participants of all skill levels. The data-set is quite small, so it is hard to make any clear conclusions, but we are satisfied with the results as of now. We saw an increase in new reporters, as well as more diversity in assets that were checked. This aligns very well with our goal to get a more thorough view of the quality of each service, as well as our security posture as a whole. By basing our plans and actions on this data, we can make educated decisions for better and more efficient work towards our goal of creating a safer experience for all our users.

How did moving to HackerOne impact the program?
To summarize the points above, moving to HackerOne allowed for an increase in participating reporters, as well as valid reports. It also resulted in a wider array of our services being inspected and tested. This closely aligned with our goals for moving to HackerOne indicating that it was a success, as well as a step towards achieving our future goals. We hope to see our program continue to improve, and hope that more and more hackers will keep participating in our program.

Team Changes

New Members and Roles
There has been a lot of changes to the team this year. First of all, we had a change of team leader.

We want to say thank you to Kubota-san for all of his hard work and wish him the best of luck in his new role. He started the process of moving our program to HackerOne, and it was only due to his hard work that we managed to reach that goal. 

In addition, we have added three new members to the team. They settled in quickly and have already contributed a lot to our daily operations. We hope the reporters are able to see how the program is able to improve thanks to their contributions.

We are always looking for new members, so if you are interested in Application Security or Bug Bounty operations, please send us your application. For more information on how to do so, see the links at the end of this post.

Weekend Work
Hackers never sleep, and that means that we also need to be ready to handle any potential issues, whenever they may arise. As such, we have started doing weekend shifts as part of our normal operations, in order to make sure that we can quickly resolve any issues and keep our customers safe. This is one of the changes we found necessary after having more global hackers participate, as the reports are no longer coming in during normal working hours in Japan, but can instead come at any time. While it means the team has to sacrifice some of their personal time, it is our highest priority to keep LINE users safe, and as such we are dedicated to continue improving in order to meet that goal.

The team is, and will keep on working hard and we are looking forward to all of the new challenges and reports 2020 will bring.
 

Plan Going Forward

Smoother Bounty Payment
One of our biggest areas of improvement is our payment process, nobody knows that better than our team. As such, we have set a goal to significantly improve procedures and handling of our payments in order to operate more efficiently and getting the reward to all the contributors as quickly as possible. We are currently considering how to best meet this goal, but we have several options lined up and hope to be able to significantly reduce the time it takes for us to finalize a bounty, before the end of this year.

Public Reports
In order to show our dedication to openness regarding security issues and vulnerabilities, as well as allow users and researchers to get an accurate impressions of LINE's security posture, another one of our goals this year is to start publicizing the reports we receive. We want to share it with the community and hope to see both increased trust in our services as well as increased participation in our program. We also hope this can be an inspiration for others to follow in our footsteps and contribute to a more open approach to security issues where sharing is encouraged.

While our initial publications are unlikely to reveal all the details of the report, we hope to be able to share a summary of the issue, a timeline of interactions relevant to the report, as well as the final bounty of course. Our goal is to be able to share all the information at some point, but we want to take a slow approach to doing so, making sure that we establish proper procedures and guidelines in order to preserve the safety and security of the reporters and our users throughout the process.

Better Understanding of Why an Issue Exists
We also want to improve our understanding of why an issue has made it to production. This is our final goal for this year.

While it will realistically never happen, the goal of any software or online service is to have no bugs or security issues in production. In order for us to come as close to that goal as possible, we are planning to add additional evaluation and analysis to our workflow, to better understand how we can improve and prevent issues more efficiently. We hope this will both help us give better explanations to reporters regarding why a bug exists, as well as provide better feedback and guidance to our developers. Our hope is that this will result in safer services for our users and better communication and explanations for our reporters.
 

Recruiting
Application Security Engineer
Mobile Developer - Security features
Infrastructure Security Engineer 
Incident Response Analyst
 

To learn more about LINE’s bug bounty program or to start hacking, visit https://hackerone.com/line.