Design for Office 365 Network Performance

This article presents architectural design guidelines for enterprise networks providing Microsoft Office 365 services. Office 365 has standard SaaS architectural requirements, but also several unique options to provide optimal application performance, scalability and resiliency.

Components outside of the core LAN and WAN environment can affect overall user experience. This includes desktop and other end-user devices, client software and operating environments. Performance can also be affected by configurations and policies managed by the customer administrator for Office 365.

This article focuses on the architecture of the enterprise network and design methodologies available to the network administrators to optimize the performance and scalability of Microsoft Office 365 services.

Cloud Services within Office 365

Office 365 has a broad set of cloud services with widely varying performance requirements. Applications like OneDrive and Outlook transfer data in the background. The volume of data can be massive but has a minor affect on user experience directly. Skype for Business provides voice and streaming video and has high requirements for network performance. Network packet drops, or latency issues affect voice quality directly creating starkly bad user experiences.

Microsoft’s minimum performance requirements for Skype for Business is:

Metric Target
Latency (one way) <50ms
Latency (RTT or Round-trip Time) <100ms
Burst packet loss <10% during any 200ms interval
Packet loss <1% during any 15s interval
Packet inter-arrival Jitter <30ms during any 15s interval
Packet reorder <0.05% out-of-order packets

Some applications require network latencies as low as 25 milliseconds. Office 365 has become a large portion of WAN traffic with over 50 applications and services. Many of these services are business critical where service outages and poor performance affect businesses directly.

The main topics covered in this article are:

  1. Optimizing network performance to the Office 365 access points
  2. Egress Data Connections To Closest Network Entry Point
  3. Bypass Customer Premise Edge Security

1. Optimizing Network Performance to Office 365 Access Points

Once Office 365 traffic is handed off to the Microsoft Global Network access point, the data is intelligently routed using advanced high-speed networking technologies. Network performance issues are often found in the network segment between the customer’s network edge and the Microsoft network.

Enterprise to Office 365 Network Data Path

The customer is responsible for this network segment. The design criteria for Office 365 networks include:

There are several network solutions available to provide customer network connectivity to Office 365 cloud services:

Direct Internet Access to Office 365

The most common method to connect to Office 365 services is by directly routing traffic over the Internet. Often the performance of this network depends on the Internet service provider. This network segment should be monitored for both performance and other issues. Refer to the section “Troubleshooting and Monitoring WANS.”

Even if the connection between the customer network and Office 365 does use this method, mobile users and potentially branch office locations will use this direct Internet route.

Network design options include:

Customer WAN Extension

Customer WAN infrastructures can be extended to ISP or co-location facilities which have direct peering relationships with Microsoft. These facilities can provide direct carrier interconnections to Microsoft.

An example configuration would be for the customer to deploy a Multiprotocol Label Switching (MPLS) circuit to an ISP providing a direct handoff to Office 365.

This would require the customer to:

The customer would be able to guarantee network performance and resiliency for business-critical Office 365 services.

ExpressRoute Peering with Microsoft Azure

Microsoft ExpressRoute® is similar to the WAN extension method above but is provided by Microsoft as a service. The network service provider would provision this direct connection to Microsoft and handoff the network to your premises, similar to other network circuits.

SD-WAN and Hybrid WAN

Advanced networking technologies are available with Application Delivery Controllers and SD-WAN solutions. These solutions provide advanced and flexible WAN architectures. Some of the network features include:

2. Egress Data Connections To Closest Network Entry Point

The Microsoft Global Network has a large number of network entry points geographically distributed world-wide. To provide the best performance for Office 365 services, customers should connect to the nearest Microsoft Network Point of Presence (POP). Once on the Microsoft network, traffic is intelligently routed through the high-speed Microsoft Global Network.

Office 365 services use GeoDNS lookup and other technologies to determine where DNS requests are coming from and redirect client traffic to the nearest geographically located Service Front Door.

Office 365 clients use local DNS servers, which forward requests to Microsoft DNS servers. The Microsoft Office 365 DNS servers identify the geographic location of the client, and direct client traffic to the closest Office 365 services front door.

Customer DNS servers must reside on the same network subnet as the clients. If the customer’s DNS servers reside far away from the clients, traffic will be routed in a suboptimal path.

The customer DNS servers should either:

  1. Best — be a local DNS server with root hints. This server will reside on the customer’s network and provide the exact customer network.
  2. Good — Forward to ISP DNS servers which reside on an ISP network segment close to the customer network.
Note: Many DNS providers with remotely located DNS servers will cause Office 365 services to route traffic poorly which may significantly reduce performance. Similar algorithms are used with GSLB technologies.

3. Bypass Customer Premise Edge Security

As traffic and the number of devices grow, typical security infrastructure and NAT implementations can become bottlenecks. Security devices may suffer performance degradation while inspecting encrypted traffic. Steering Office 365 traffic around the edges of customer premise equipment reduce as much as 80% of the load on the egress and edge security systems.

The benefits of steering Office 365 traffic directly to the Office 365 access point is:

Microsoft Office 365 Bypassing Edge Security

Each individual user creates dozens of long-lived connections to the Office 365 hosted environment. Large documents frequently transfer to and from the cloud. NAT servers are stressed similarly.

Note: Microsoft recommends a limit of 2000 Office 365 clients on NAT servers for each public IP address.

Office 365 client network traffic to cloud services traverse the edge network and security systems. This stack of edge systems is complex, expensive and must be highly reliable. Office 365 services include security services which may be redundant and can be routed directly to the Microsoft network securely, bypassing many of the enterprise edge systems.

Note: Microsoft recommends direct network connectivity to Office 365 services for faster and low-latency performance.
Microsoft Office 365 Traffic Steering

In the diagram above, Office 365 traffic detected and steered directly to the Office 365 network access point. Other Internet traffic is sent to the edge security systems for inspection. Office 365 provides similar security processing and is provided by Microsoft.

This network architecture is described here: Enterprise Network Architecture

Steering Office 365 traffic around edge CPE systems requires:

  1. Identifying Office 365 Traffic
  2. Adding a network load balancer (ADC) system to intelligently route traffic

Identifying Office 365 Traffic

Office 365 network traffic is encrypted but can be identified by the destination URL or IP address in the HTTPS and IP headers. Microsoft publishes the current Office 365 network endpoint URLs, IPv4 and IPv6 addresses and TCP/UDP ports. This identifies every network access point for every Office 365 service.

This endpoint data is downloaded and used to configure the traffic steering load balancer to route data destined to Office 365 endpoints directly. All other traffic is routed to the edge security systems, the same as before.

The Office 365 data changes over time and must be downloaded and reloaded into the load balancer to refresh the endpoint data. Here is the current Office 365 URLs and IP address ranges.

Below is a small sample of this data.

Skype for Business Online and Microsoft Teams

ID Category ER Addresses Ports
11 Optimize Required Yes 13.107.64.0/18, 52.112.0.0/14 UDP: 3478, 3479, 3480, 3481
12 Allow Required Yes *.lync.com, *.teams.microsoft.com, teams.microsoft.com, 13.70.151.216/32, 13.71.127.197/32, 13.72.245.115/32, 13.73.1.120/32, 13.75.126.169/32, 13.89.240.113/32, 13.107.3.0/24, 13.107.64.0/18, 51.140.155.234/32, 51.140.203.190/32, 51.141.51.76/32, TCP: 443, 80

Troubleshooting and Monitoring Office 365 Traffic

The network between the customer network edge to the Microsoft network edge is typically provided by an Internet Service Provider (ISP). This portion of the Office 365 access network is outside the control of both the customer and Microsoft and should be established and monitored carefully.

Express Route if a service available which provides direct leased line connections to the Microsoft network. Express Route provides a superior solution with Service Level Agreements available. This solution has a higher operating cost.

This network connection can be tested and monitored using several network tools including:

This network segment should be monitored for performance metrics including:

1. Round time delay and latency.

a. Using utilities like PING/psping will identify general performance problems but will not provide much more information for diagnostics

2. Network routing issues like “hairpins” and routing loops

a. Once problems are identified with PING, issues within the ISP networks can be identified using a network trace route utility.
b. Run a tracert command to a Microsoft service. This utility will show each hop in the network displaying the details of the entire network route. Each hop will be listed with the round-trip time. Analyze the route data and look for:

i. Long delay times between network hops
ii. Routing paths to distant geographic destinations.
iii. Odd route paths

3. Stored historical performance data.

a. Network monitoring tool are available which continuously monitor network performance.
b. Collect this data to calculate a historical baseline performance
c. These tools will usually send events for performance issues

4. Microsoft Office 365 includes monitoring tools.

a. These tools test the complete path from the Office 365 client to the Office 365 cloud service. One example is the Skype Network Monitor and Assessment tool. This tool will simulate Skype traffic for 15+ seconds and monitor for network performance, packet loss and jitter

References

How A10 Networks Can Help

A10 Networks has partnered with Microsoft to design an Office 365 solution for enterprise networks. A10 Networks’ Thunder products provide a complete enterprise solution or can augment existing edge systems to offload, scale and secure Office 365 applications. Read more about our solution at A10 Networks Solutions Brief for Office 365

Key Benefits


|

December 27, 2018

About Robert Keith

Robert has 30 years of experience in IT technology development and infrastructure management. He was the founder of several infrastructure ventures including Intellivence, MaxSP, Sentrik and most recently was the CTO of Iron Networks. As CTO of Iron Networks in San Jose, CA, he worked directly with many companies in the Silicon Valley to design and architect network, security, and cloud solutions. He worked directly with Microsoft engineering in the design of their cloud architectures including storage, Hyper-V, Systems Center and Virtual Networking. He also worked directly with Hortonworks to design a Hadoop deployment and management system using CentOS and many layered software packages. READ MORE