Disaster Recovery

Indian Oil's two-tier advantage

S Ramaswamy

In 2002 Indian Oil Corporation Limited (IOCL) realised that deploying solutions on a centralised architecture has advantages and disadvantages. Any failure of the centralised database server would bring IOCL’s business to its knees. This was the point when IOCL decided to improve the availability of its SAP R/3 application by designing and deploying a two-tier disaster recovery (DR) architecture.

As S Ramaswamy, GM, IS, IOCL informs, “Data storage and back-ups on servers form a crucial part of our mission-critical operations that need to be addressed by an advanced architecture and related support services.”

Among the world’s largest

IOCL’s SAP R/3 installation is one of the largest of its kind in the world. It hosts over 5,000 simultaneous users from 600 locations. The systems and services operate on a centralised architecture and rely on a centralised database system. This helps IOCL users across the country to connect to centralised database servers.

Today, the deployed architecture’s sole objective is to maintain data integrity, DR and critical business management. “The deployment complements business continuity (BC) at IOCL with minimal downtime. Data losses are eliminated or minimised at the very least,” says Ramaswamy.

For every eventuality

Potential disruptions and disasters have been categorised and documented. Disasters are categorised into local, logical data and site failures. Local failures include CPU faults as well as failures in disc and array, LAN, SAN, software and utilities. Logical data failures include system crashes due to software bugs, data corruption, accidental deletion and the application of incorrect patches. Site failures include power and telecom outages, acts of war or terrorist attacks, and human errors.

Based on disaster criticality, the SAP R/3 services are provided from four different centres. Under no-disaster conditions, the services operate out of IOCL’s Corporate Data Centre (CDC) for the primary production system. The CDC is located at the Indian Oil Institute of Petroleum, Gurgaon.

The R/3 production environment constitutes a database server with an R/3 central instance server and 15 R/3 application servers. The deployment also includes various enterprise-class storage sub-systems and a tape library over a high-speed LAN and SAN.

The database server and the central R/3 instance run on a HP Superdome 9000-class machines with 18 HP PA-RISC 8700 series CPUs and 24 GB of main memory. This machine is connected to the HP XP1024 storage sub-system and HP ESL tape library over the SAN through an HP FC-64 SAN switch. The database server currently runs on Oracle Database Server Version 9.2.0.5.

Individually, every SAP user is connected to the production system. This is achieved through any of the 15 SAP R/3 application servers. These are deployed on 13 HP Series 5405 servers (3 CPUs, 6 GB RAM) and two HP series 7410 servers (8 CPUs and 24 GB RAM).

The Corporate Communication Centre acts as the primary communication hub under normal conditions. The centre is also located at Gurgaon, but in a separate building at a distance of 300 metres from the CDC. A redundant baseband and radio frequency (RF) solution is also provided in addition to cable connectivity. This centre forms the backbone for various e-commerce applications, SAP, FTP and e-mail.

Finally, there is the Metro Disaster Recovery Centre (DR1) to be used as a production system during a localised disaster at the CDC. This centre also hosts five HP L-class servers (three CPUs and 6 GB RAM) running as application servers. It is configured for the primary production environment. IOCL’s development and quality server hosted at DR1 is on an HP L-class server (three CPUs and 6 GB RAM). Cisco’s network node manager and HP OpenView software are used to manage the communications centre.

Twice the strength

The two-tier recovery architecture aids faster and consistent recovery. It is also expected to support communication and SAP R/3 services during natural and man-made disasters.

DR1 operates from the communication centre within the complex, and lies about 300 metres from the production data centre.

The Main Disaster Recovery Centre is based at the Sanganer terminal in Jaipur. It is located about 250 km from the Gurgaon facility, and is known as the Business Continuity Centre.