As your business grows, you encounter new challenges. It becomes clear just how mission critical the tools your teams rely on are - often as the size of those teams means your current setup has become unstable.
This is why enterprise businesses are turning to Atlassian's Data Center options to give their tools the stability, high availability and reliability necessary for teamwork at scale.
At some point in your Atlassian journey you’ll likely find yourself needing to scale your Atlassian applications to a wider group in your organization. Take this quick quiz to find out if you’re ready to scale with Data Center.
Cerner, the leading U.S. supplier of healthcare information technology, had three federated instances of JIRA Software Server with thousands of developers using each instance every hour of every day around the globe. JIRA Software quickly became mission-critical and every minute of downtime or performance degradation made it more difficult for Cerner team members to support their customers. They needed a solution that provided high availability.
In the fall of 2015, Cerner chose to upgrade from Server to Data Center so they could cluster multiple active servers and provide users with uninterrupted access to JIRA Software. This wasn’t just critical at the time, but knowing that they were going to add several thousand more users in the coming year, they needed a solution that would scale with them.
Using Zabbix and Splunk to monitor their JIRA instances, Cerner was able to identify one area that needed to be addressed immediately if they wanted to provide true high availability: REST API abuse.
Their log analysis showed that team members were using the REST API to get real-time status updates – so whether teams knew it or not, they were pinging JIRA Software instances every single second. Cerner didn’t want to restrict users from creating custom dashboards or self-serving the data they needed, but it was obvious that they had to do something different.
With a multi-node cluster they could intelligently distribute traffic by dedicating one node solely to external REST API requests. Cerner also wanted to guarantee that all external requests went to this dedicated node because having users manually change the domain to an IP address, or another domain, wasn’t reliable.
Cerner needed the Data Center configuration to ensure all external REST API requests were routed away from other traffic. They planned to have four nodes in their cluster behind a load balancer with each node performing the following services:
Originally, they thought the best option would be to use the load balancer to route all requests with ‘/rest’ to the REST API node. However, after some testing they found that the REST API was also being used throughout JIRA Software, including the login page, so leaving it to ‘/rest’ would mean they would still be mixing REST API traffic with normal usage.
They then found that they could isolate REST API requests by looking for ‘/rest’ in each request AND by looking at where the request originated using the HTTP referrer header. If a person was trying to login to JIRA Software or was already using JIRA Software, they would get directed to or remain on a Normal Usage node. Otherwise, if the person or bot was requesting the REST API, they would get directed to the REST API node. After a few rounds of testing, Cerner went live with JIRA Software Data Center in October 2015.
Within their first week of implementing the proposed Data Center configuration, Cerner was seeing 4 times the amount of traffic on the REST API node as on the other two nodes. Response times are faster, CPU utilization has decreased across their non-admin nodes compared to a single server instance, and they haven’t seen a single unplanned outage in 2016, all while scaling JIRA Software to thousands of new users. With the move to Data Center, Cerner experienced:
Cerner needed to make sure that as they continued to add users that application response times were maintained or improved. This re-architecting proved that Cerner was able to reduce their response time by nearly half, from 150ms to 80ms. Even at peak traffic times – looking at page loads specifically – response times remained steady.