Skip to content
11:11 Systems
Rethink Connected
11:11 Systems11:11 Systems
  • Why 11:11
    • Submenu
      • Column 1
        • Cloud Console
          Cloud Console
          Compliance
          Compliance

      • Column 2
        • Global Regions
          Cloud Regions
          Catalyst
          Planning and Assessment

      • WHY CHOOSE 11:11
      • Overview
      • Leadership
      • News & Media
      •  
      • Careers
      • Technology Partners
      • Customer Stories
  • Products & Services
    • Products & Services
      • CLOUD
      • Cloud Overview
      • Public Cloud
      • Private Cloud
      • Object Storage
      • Cloud Labs
      • Colocation/Bare-Metal
      • BACKUP
      • Backup Overview
      • Veeam Backup
      • Microsoft 365 Backup
      • Managed Backup for Cohesity
      • DISASTER RECOVERY
      • DRaaS Overview
      • DRaaS for Veeam
      • DRaaS for Zerto
      • DRaaS for Azure
      • Autopilot
      • SECURITY
      • Security Overview
      • Continuous Risk Scanning
      • Managed SIEM
      • Managed EDR
      • Managed Firewall
      • CONNECTIVITY
      • Connectivity Overview
      • SD-WAN
      • Multi-Cloud Connect
      • Managed IP
  • Solutions
    • Solutions Submenu
      • INDUSTRY
      • Education
      • Financial
      • Government
      • Healthcare
  • Partners
    • Partners Submenu
      • Overview
      • Become a Partner
      • Partner Portals
  • Resources
    • Resources Submenu
      • Events
      • Webinars
      • News & Media
      • White Papers
      • Podcast
      • Data Sheets
      • Customer Stories
      • Innovation Blog
  • Support
    • Support Submenu
      • Contact Support
      • Product Documentation
      • API Documentation
Search:
  • Console Login
  • Contact
Header Right Menu
Free Trial
  • Why 11:11
    • Cloud Console
    • Compliance
    • Cloud Regions
    • Planning and Assessment
    • WHY CHOOSE 11:11
    • Overview
    • Leadership
    • News & Media
    • Careers
    • Technology Partners
    • Customer Stories
    • Blog
  • Products & Services
    • CLOUD
    • Cloud Overview
    • Public Cloud
    • Private Cloud
    • Object Storage
    • Cloud Labs
    • Colocation/Bare-Metal
    • BACKUP
    • Backup Overview
    • Veeam Backup
    • Microsoft 365 Backup
    • Managed Backup for Cohesity
    • DISASTER RECOVERY
    • DRaaS Overview
    • DRaaS for Veeam
    • DRaaS for Zerto
    • DRaaS for Azure
    • Autopilot
    • SECURITY
    • Security Overview
    • Continuous Risk Scanning
    • Managed SIEM
    • Managed EDR
    • Managed Firewall
    • CLOUD CONNECTIVITY
    • Connectivity Overview
    • SD-WAN
    • Multi Cloud Connect
    • Managed IP
  • Solutions
    • INDUSTRY
    • Education
    • Financial
    • Government
    • Healthcare
    • Column 2
  • Partners
    • Overview
    • Become a Partner
    • Partner Portals
  • Resources
    • Events
    • Webinars
    • News & Media
    • Whitepapers
    • Podcast
    • Datasheets
    • Customer Stories
    • Innovation Blog
  • Support
    • Contact Support
    • Success Center
    • API Documentation
  • Contact
  • Console Login
  • Free Trial
Tags: Disaster Recovery
Author: Mike Mosley
Date: November 9, 2018

Partial Failovers and the Hidden Challenges

Editor’s Note: As of January 2022, iland is now 11:11 Systems, a managed infrastructure solutions provider at the forefront of cloud, connectivity, and security. As a legacy iland.com blog post, this article likely contains information that is no longer relevant. For the most up-to-date product information and resources, or if you have further questions, please refer to the 11:11 Systems Success Center or contact us directly.

In the world of DRaaS, partial failovers have become a very popular topic of discussion. Essentially, customers want to know that if needed, they will be able to failover a subset of their production environment. While partial failovers are possible, there are several challenges that need to be considered ahead of time.

First, let’s clarify the difference between full and partial failovers. The fundamental goal of a DRaaS solution is to recover your production environment after a total site disaster. This includes bringing your systems back online and enabling network connectivity to the outside world or branch offices. Below is a diagram of what a standard DRaaS setup may look like between a customer’s premise site and 11:11 Systems.

The source or premise site will replicate to 11:11 over a replication VPN tunnel to an 11:11 firewall. This replication firewall only handles replication traffic and is independent of the failover environment and failover networking. When a failover is initiated, the replica servers boot up at the 11:11 site and use the failover firewall for all network connectivity. The important thing to note in the diagram above is that there is no connectivity between the source premise servers and the replica servers. In this scenario, you should assume that the premise site is offline and connectivity is not needed. Additionally, the replica servers that boot up at 11:11 should be exact mirrors of their source counterparts, as they will have the same hostname and IP address.

The above example details the flow of a full-site failover. However, what happens if the full site is not impacted and only one server or a subset of servers need to failover? Some organizations have made requirements that they are able to granularly failover their environment. For instance, let’s change the above diagram and say that only A, B, E and F have become degraded and need to failover.

In this scenario, we are able to recover the impacted servers without having to failover the entire environment. You will notice that in the diagram below, there is a new tunnel enabling connectivity between your recovered servers and your production servers that were not affected. Here is where this whole process gets tricky, as there are several options, each having its own caveats.

Review Your Use Case for a Partial Failover

There are many reasons why an organization may want to prepare for a partial failover along with preparing for a full site emergency. You may have physical resources along with your virtual environment and want to protect yourself when only the virtual resources are impacted. Or, you may want to protect yourself from unexpected host failures or an accidental VM deletion. You might also want to review communication between your premise data center and other off-site or branch locations. You will also want to confirm if your servers are being backed up and if so how often? Are they saved locally, off-site or both?

Your goal initially should be to get a good understanding of exactly what changes or steps are needed during a partial failover. You may find that the labor needed during a partial failover scenario outweighs just restoring a server from a backup. Enabling communication for the partially failed over servers may take longer to fully configure than you are able to tolerate. Mapping out the partial failover process, including the internal or external changes needed, expected time to complete, and overall difficulty may help in figuring out an alternative method. Unfortunately, planning for a partial failover includes overcoming challenges and obstacles that are not as problematic when failing over a full environment.

Partial Failover Challenges

The main challenge with running a partial failover is enabling communication from your DR site back to production. In the above partial failover diagram, the A, B, E and F servers that failed over would need to communicate back with the other unaffected source servers. In typical DR setups, the networking at the failover site would mimic the source site. So if the “A” VM has IP address of 192.168.1.10 at the source site, it will have the same IP address after failover. This ensures that server communication and any AD/DNS configuration remains the same for your environment after a failover.

However, if the failover and production subnets are the same, you are not able to create an IPSec VPN tunnel between the sites. This is because creating a VPN between two sites using the same subnet makes routing impossible. When server A fails over with IP address 192.168.1.10 and needs to connect to server C with IP address 192.168.1.13, the firewall is unable to determine if traffic from server A should stay local or route across the VPN tunnel. Because of this confusion in the routing, the VPN tunnel will never establish or successfully pass traffic. Essentially, the simplest method to create a connection between two sites becomes impossible.

It is possible to create a VPN tunnel between the failover and production sites if they are not using the same subnets. So instead of the A server failing over with IP address 192.168.1.10 it may use something like 192.168.2.10. This would be true with all servers, so if any server fails over, it would change its IP to have a 2 in the third octet. While this allows for VPN connectivity, you then will have to account for additional caveats.

First, when server A boots up at the recovery site with its new IP address, it will have connectivity back to your Active Directory and DNS server. This means server A’s IP address will be updated in your production AD and DNS infrastructure and then replicated to remote AD/DNS servers. While this works during a live scenario, it makes testing a bit harder. With most DR solutions, you are able to test a failover without having to impact or take down any production servers. However, with this method, you could potentially have server A online at the DR site and the production site at the same time. While they would have different IP addresses, their hostnames would conflict in your AD/DNS infrastructure. What’s worse, production changes or activity from the other production server may get routed to replica server A. If you end the test, those changes will be erased and you may lose hours of production work or updates. Essentially, this method causes a much more intrusive testing or failover method and needs to be heavily analyzed before testing.

This setup also becomes complicated when determining how to configure your DR replication. Typically, you have to configure the network and IP address used by a server once it has failed over. Some solutions allow you to use a different network between a live failover and a test failover, but you are not really able to prepare for a full versus partial failover. So, if you have your replica jobs or failover environment set up for a full-site failover but a partial is needed, you may need to manually make changes before, during or after the failover. This can increase the possibility of mistakes and lengthen your failover process if it is not fully documented.

If setting up a VPN tunnel or changing IPs seems too problematic or complicated, you may want to look at alternate methods, such as public communication or a quick failover and failback. Public communication would involve setting up a public IP that NATs to one of your replica servers. So, if server A is failed over, your production servers can communicate with it over the public IP address. However, this could involve you needing to first make public DNS record changes which can take time to populate across the public network. You also may have internal scripts, applications or processes that refer to server A by IP address. If that IP address changes, this process may break. Communicating over the public internet is drastically less secure and may not be a viable option for many companies.

If you can tolerate downtime for an hour or more, you could also consider failing over and then failing back to your production environment. This can be useful if you want to protect against an accidental VM deletion or if it becomes corrupted from malware. Basically, you could fail the server over to the DR location without connectivity back to production. If you are able to confirm the data needed is on the server and uncorrupted, you could enable reverse replication and failback to your production data center. This may take longer than restoring a VM from a backup but it may have more up-to-date data available. While this method takes longer to make the data accessible, it also removes any changes or work needed to establish communication between the sites.

Looking Forward and Data Center Extension

Data center extension has become a more popular setup as it essentially creates a layer two connection between the recovery and production data centers. This overcomes the challenges with the VPN as routing is not needed in this setup. While this has been an option in the past, it has become a bit more difficult as it was usually a physical cross-connect or direct circuit installation. However, as the DR industry and providers continue to grow and innovate, virtual and appliance options are becoming available. Of course, data center extension can involve the same or unique caveats as explained above.

Partial failovers will continue to be a requirement or major concern for companies as it is important to be protected for any disaster scenario. It is important to plan, diagram and test your failover process as often as possible to ensure you are prepared for disaster. If you have any questions about your DR solution, feel free to contact 11:11 for assistance or advice on your recoverability.

Category: DRaaSBy Mike MosleyNovember 9, 2018
Tags: Disaster Recovery

Author: Mike Mosley

Mike Mosely is a cloud engineer at 11:11 and has worked at the company for over 3 years. He holds a number of VMware certifications including VCP5 as well as the Veeam VMCE certification. Mike works closely with customers to build cloud solutions that fit their requirements.

Post navigation

PreviousPrevious post:Is Nephophobia Hindering Your Disaster Recovery?NextNext post:Simplified Data Protection for Office 365

Related Posts

Preparing for 2023 with 11:11 Systems: IT Trends in Security, Cloud, and More
February 1, 2023
11:11 Systems Wins 2022 Backup and Disaster Recovery Award from Cloud Computing Magazine
January 25, 2023
How businesses can respond to IT disruptions during the holiday season
How businesses can respond to IT disruptions during the holiday season
January 4, 2023
The CloudBytes Podcast: Season 3 is Here!
December 2, 2022
Celebrating Get To Know Your Customers Day
October 20, 2022
Experts: Prepare for Busy 2022 Hurricane Season
July 1, 2022
PRODUCTS & SERVICES
  • Cloud
  • Backup
  • Disaster Recovery
  • Managed Security
  • Connectivity Solutions
  • Compliance
COMPANY
  • Why 11:11
  • Customer Stories
  • Careers
  • Leadership
  • Technology Partners
  • News & Media
  • Contact Support
CLOUD REGIONS
  • North America
  • EMEA
  • APAC
CONNECT
  • LinkedIn
  • Twitter
  • Facebook
  • Youtube

© 2023 11:11 Systems Inc., All Rights Reserved | Privacy Notice

Go to Top
PRIVACY POLICY AND COOKIE CONSENT
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Manage options Manage services Manage vendors Read more about these purposes
View preferences
{title} {title} {title}
PRIVACY POLICY AND COOKIE CONSENT
To provide the best experiences, we use technologies like cookies to store and/or access device information that allows us to process data such as browsing behavior. Not consenting or withdrawing consent, may adversely affect certain features and functions. By clicking Accept, closing this message, or continuing to browse, you consent to these technologies and accept our Privacy Notice.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Manage options Manage services Manage vendors Read more about these purposes
View preferences
{title} {title} {title}