Focused on Recovery-as-a-Service

SunGard Managed Recovery Program (MRP) service continues to gain momentum. Last week, another customer came through the MRP on-boarding test process. Their comments and findings provide a “before” and “after” picture of what organizations working with this exciting new service from SunGard can expect:

• The organization reported “significant” improvements with “tremendous value” while also “reducing the number of people we must send in the event of a disaster”.
• A cross-discipline approach now in place at SunGard was identified as speeding problem resolution during test.
• Improved communication between the SunGard test team and the customer’s DR team was noted, including the practice of hosting a conference call that kept all members of the team in constant communication through the test exercise.
• SunGard’s new role of Service Deliver Manager (SDM) was cited as being key in providing a single point of communication, resulting in the “smoothest test ever” from a communications and coordination standpoint.
• Upcoming participation by SunGard in the organization’s weekly change control meetings is anticipated to be a “very positive” measure towards enabling SunGard to lead recovery for the organization. They note this as being critical because most of their technical team is located far from the SunGard Philadelphia recovery site location.
• The organization’s goal is to eliminate all requirements for their team to come on site, either during test or recovery.

The experience reported by this organization mirrors similar experiences of other organizations coming through the SunGard MRP test process. Examples of these experiences were recently shared at the Gartner Data Center Conference during a panel discussion led by Kerwin Myers, Sr Director of Product Management for MRP service at SunGard. Also participating on the panel discussion, were two senior Service Delivery Managers (SDMs).

If you have questions about MRP service and how it can solve disaster recovery challenges for your DR team, submit your comments and questions. The team at SunGard will get back to you promptly.

In the consideration of cloud-based recovery services, organizations need to understand more details of how cloud platforms are implemented for recovery. In this blog entry, I walk through the details of how SunGard has implemented a portion of its cloud recovery platform in support of the Recover2Cloud for Server Replication service.

Understanding a Virtual Data Center (VDC)

Within the cloud, one of the considerations for recovery is to ensure secure access to protected data. SunGard implements a dedicated VDC in the cloud using dedicated firewall protection for each client.  VPN connection is established between the firewall and a firewall in the client’s production location.

Within the targeted cloud, there is also a dedicated live virtual machine (VM) for each customer. This VM receives the replicated data, and writes the data as dormant VMDK files.

At time of disaster, SunGard recovery automation launches the VMDK files to bring the VMs live within the VDC. This is done using server processing capacity which is allocated on-demand within the cloud. SunGard provides public IP addresses for those VMs which are public Internet facing. SunGard also provides multiple VLANs for the recovered VMs. All of this is done behind an additional dedicated virtual firewall, to ensure secure access to the recovered application environment.

Failover At Time Of Test (ATOT)

Organizations require the capability to test failover without having to shut down their production environment. This avoids testing in off-hours, such as overnight and on holidays.

To accommodate this requirement, SunGard implements patent-pending sandbox testing using a dedicated test VDC which is separate from the recovery VDC in the SunGard cloud implementation. The basic idea is that prior to test, SunGard instantiates a virtual firewall, creates the required VLANs, sets up the firewall policies between the test VDC and the required servers that will be recovered for test.

An organization working with SunGard can then select the recovery points based on the test goals, which can be either to test crash-consistency of the data or application consistency.

When recovery is triggered, the dormant VMDKs go live behind the newly instantiated virtual firewall. The VMs are assigned appropriate IP addresses, VLANs and port groups as part of the recovery process. The order in which servers are restarted is configured and automated as part of the recovery process, as well, to ensure that the interdependent applications environment is restarted quickly and correctly without risk of human error.

Once the recovery is complete, organizations can use a VPN client to connect to the sandbox VDC test environment. This enables remote test. Remote administrators can even choose use this method as a better means to test for OS or service pack maintenance upgrades.

When organizations have a hybrid environment, they can also connect to hot site or recovered workgroup environments along with the VDC.

How Secure is the Cloud?

SunGard implements enterprise-class security as part of the recovery cloud platform. A Recover2Cloud VDC looks like a remote office network to a client, accessible using secure IPsec VPN tunnel connections. At network layer 2, every R2C customer VDC is isolated from other VDCs in the cloud by using separate VLANs. At network layer 3, each customer VDC is securely protected using dedicated firewalls.

The move from tape as a basis for recovery, to replication as is provided in the SunGard Recover2Cloud for Server Replication service, offers many benefits. Benefits include:

• Faster recovery time objectives (RTOs) – recovery is guaranteed by SunGard Recover2Cloud service at sub-4 hours for the entire mission-critical applications environment that is protected
• Improved recovery point objectives (RPOs) — offered at near-zero with Server Replication capabilities
• Transformed recovery testing — avoiding the common costs and burdens, such as lengthy time of test, tape transport and loading, problems with not having enough time to complete test

Yet organizations sometimes seem to hesitate in adopting modern recovery methods. In an upcoming webinar scheduled for 11.30, SunGard will interview two professionals at Acacia Federal Savings Bank who have made the transition to replication-based recovery with success. Acacia will share their experiences in the webinar first-hand, to help other organizations in evaluating the SunGard Recover2Cloud service.

Some of the insights which attendees of the upcoming webinar can expect, include:

• How replication-based recovery compares to tape-based
• Why SunGard’s patent-pending service implementation completely transformed Acacia’s test process
• How replication complements Acacia’s backup-to-tape strategy

Interested to attend and ask your own questions? You can register now for the live event:

• Anantomy of a Cloud-Based Recovery
• November 30, 2011
• 10 am Pacific/ 1pm Eastern

In addition to speaking with Acacia representatives, a demonstration of the Recover2Cloud Server Replication product will also be offered during the webinar.

Last week, SunGard Availability Services announced General Availability (GA) of a new service called Managed Recovery Program (MRP). This service offers to help ensure the availability of applications by supporting management of the entire recovery lifecycle using a systematic approach for SunGard customers. As part of the approach, root-cause problems which undermine recovery are identified and addressed on an on-going basis.

During analyst briefings, what was the consistent comment heard by the team? That while others may talk about change management and other common root-cause problems which undermine recovery, SunGard is the only vendor with a packaged service, at General Availability, with real customers using it.

During briefings, SunGard Sr Director Kerwin Myers shared details of the experiences of early adopters of MRP services. Some of the comments shared by Kerwin include:

* Organizations with recovery scripts already in place, are often surprised to find significant gaps in those scripts which are revealed in on-boarding tests

* Lack of change management of recovery procedures is one of the most significant root causes which MRP is designed to address

* MRP customers report appreciating the benefits experienced from working with an assigned SunGard Service Delivery Manager (SDM), in particular, as the SDM serves as a counter-part to the organization’s DRC and helps to re-enforce and strengthen that role

Review of MRP service has been strong. You can read what’s being said in this round-up of coverage:

* “SunGard Availability Reveals Recovery Plan for Large Enterprises” Posted Nov 15, by Bruce Hoard at Virtualization Review

* “SunGard Announces New Managed Recovery Program” posted Nov 9, by Loiver VanDervoort at MSPNews

* “SunGard: Managed Recovery Progam for Cloud and On-Premise” posted Nov 10, by Nicholas Mukhar at MSPmentor

Interested in learning more? Kerwin Myers and the SunGard MRP team will be presenting a session on overcoming root-cause problems which undermine recovery, at the upcoming Gartner Data Center Conference being held Dec 5-8 in Las Vegas. Meet Kerwin and several of our SunGard SDM experts while you’re there!

Next week on Nov 9, Information Week and Network Computing host a virtual event called “Virtualization Anywhere.” At this event, sponsoring vendors will provide information about their products and services — and how they can help you to address challenges with managing virtual systems and infrastructures. An online virtual event emulates in-person trade shows, but are far more convenient  and are usually free to attend.

If you’ve never attended a virtual event like this one, then you’ll be interested to know that each vendor is available in a virtual booth. You navigate an online map which emulates a typical event show floor, and you decide which vendor booths you would like to visit. When you visit, you can pick up digital copies of product literature. And, most importantly, you can chat in real-time with vendor representatives.

In addition to the virtual show floor, virtual events also feature a series of live webinars which you can attend — emulating “main-tent” style presentations which occur at in-person events.

SunGard Availability Services is a platinum sponsor at next week’s event, and we hope you come to see us there! Our speaker, Ramanan Shanmugam, is Sr Director of Product Management for the Recover2Cloud suite of services. Ram will cover the top five considerations for raising service levels with tiered recovery services and cloud services, with insights learned from SunGard’s track record with providing disaster recovery services for over 30 years.

Interested in learning more about the event? See the event web site, and register to attend today!

If you’re busy on Nov 9 and you cannot attend, check this blog site after the event. We will share the links to Ram’s recorded presentation following the event.

A common concern often voiced by administrators who need to test applications and systems recovery, is that performing the test disrupts their application service. A test cannot be performed for recovery unless the production application is taken down. This is often the case, because the recovery site IP addresses (and machine name aliases in some cases) disrupt connectivity at the production site. To avoid this, the production applications must be taken offline during the test and then reconnected once the test is successfully complete.

Depending on the recovery process, the disruption period can last a long time. Annual disaster recovery testing can take days to perform. As described in previous entries on this blog site, disaster recovery testing can be especially time-consuming and challenging. Many organizations report not being able to get past “best-effort” testing as a result.

New Patent-Pending Capability from SunGard Enables Production Services to Keep Running Throughout Recovery Test

SunGard has introduced a new patent-pending test process designed to keep application services online throughout a remote-site recovery test. The process includes the creation of a Recover2Cloud(sm) virtual data center (VDC), and leverages cloud platform technology. In this process, all of the resources required to perform a recovery test are allocated on-demand. This includes management of firewall and security systems. Testing proceeds using a technology “sandbox” which is isolated from the network runnning the production application service.

SunGard's Patent-Pending Sandbox Test Process

Using this new approach, organizations can now keep their production application environments up and running while using the SunGard test VDC for validating the viability of their Recover2Cloud configurations and other scenarios.

Also, because the approach leverages replication software built into the Recover2Cloud service suite, each customer can rest assured that their exact production environment is duplicated as a replica within the test VDC sandbox. This replica environment helps ensure that the recovery test is valid and reliable, to a degree not previously achievable without taking production systems off-line.

The test VDC approach does not utilize the Recover2Cloud disaster VDC resources. This helps ensure that in the unlikely event that an actual disaster is declared during test the recovery process to the Recover2Cloud disaster VDC proceeds without delay.

—-> By Kelly Baig
Last Spring, Forrester published some interesting data from a late 2010 survey they conducted with Disaster Recovery Journal of 200 DRJ members. In that survey, 61% of respondents reported that they felt “very prepared” or “prepared” to recover data centers in the event of a site failure. This is an increase from 52% in response to the same question in 2007.

Yet, in the same survey, test of recovery plans reportedly went down — and organizations reporting “no test at all” went up. Also, updates to test plans likewise went down.

Thorough test of recovery plans and procedures is widely known as being the critical success factor in enabling successful recovery when you need it. The discrepency between levels of confidence and sufficient test reported in the survey, begs the question: why are so many organizations confident of their recovery?

Are Test Practices Improving in 2011?

This week, Rachel Dines — the Forrester Analyst who worked with DRJ on the 2010 survey — posted a new report entitled “Disaster Recovery Exercises Fall Short Of The Finish Line.”

The title and 10 useful tips for testing which Rachel identifies, seem to indicate that test practices have not improved in 2011 for most organizations.

In fact, for most organizations, completing test for an entire disaster recovery plan may be relatively rare. Many times, “best-effort” tests seem to be the norm. These efforts are not sufficient to ensure recovery. Yet test costs and management burdens appear to prevent these practices from improving.

Getting Past “Best-Effort” Testing

Clients working with SunGard Availability Services have found relief from test costs and burdens, working with a new service called Managed Recovery Program (MRP). This service uses best practices defined by SunGard, to prescribe test frequency and test methods.

Each MRP customer is assigned a Solutions Delivery Manager (SDM) responsible for monitoring changes in a client’s production environment. SDMs then recommend appropriate modifications in recovery test plans, to keep up with production changes. They also prescribe test frequency and priority. At test as well as disaster, SunGard recovery experts assume the recovery burden, eliminating all of the headaches and costs while raising recovery readiness.

Adopters of MRP come through a thorough recovery test as part of their on-boarding process. These customers have been surprised in many cases, to realize that they had significant gaps in their recovery plans about which they were unaware prior to test.

Interested in learning more about MRP? Check out this recent MRP video posted on YouTube for a quick overview.