A billion-dollar Manager Services Provider, based in the US, offers multiple products and services to their clients. It provides database management and migration to Cloud, among other services. The company has been a customer of Doyensys for several years. When they connected with Doyensys to facilitate a mission-critical system for one of their clients, the team was up for the new challenge.
This client was in the fitness industry and offer products and services like weight loss and maintenance, fitness, and lifestyle changes modules. The peak business season for them starts post-Christmas and usually continues till the end of February or the beginning of March.
Problem Statement
Apart from developing a mission-critical system, the client had other requirements as well. The customer had implemented an architecture that included Oracle Real Application Clusters (RAC) and two Data Guard physical standby databases. While one was local to the production data center for recovery from operations-induced data corruption, the other was remote for disaster recovery (DR).
The client also needed support for managing costs as the customer had consolidated QA/performance test, integration test, and development environments onto the same environment that were used for disaster recovery.
Project Requirements
This project was a unique scenario as Doyensys worked for a customer for their client. The client approached Doyensys with a host of requirements. Firstly, they wanted to upgrade the system, including the hardware, operating system, grid home from 12c to 19c, middle tiers, and the network. Some of these upgrades were small but significant for the user. The final goal for the customer was to save costs.
Secondly, they wanted to enhance the performance of e-commerce applications and back-office processing while also supporting a monthly/quarterly closing. The client has a process where all the processing to close financial books globally is done within one working day.
They also wanted to build a resilient architecture to assure availability in the face of both physical and logical failures.
Lastly, and most importantly, they wanted to optimize footprint, deployment, and ongoing management costs.
Solutions
Doyensys, through its exceptional technical capabilities, was able to add value to the client’s needs by offering and implementing efficient solutions.
The team smoothly migrated financial Production and Non-Production ERP systems from one data center to another. Secondly, we also offered a new environment that makes extensive use of Oracle Database System and Maximum Availability Architecture. As a result, the client benefited from a 24X7, high performance, and comprehensive data protection in a consolidated environment characterized by a high volume of mixed workloads (OLTP, reporting, and batch) and demanding service level expectations.
To maximize the use of their computing assets, the team implemented a cost-effective architecture by creatively using Data Guard standby systems for multiple purposes and incorporating low-cost storage.
System Monitoring
To ensure uninterrupted functioning, the team at Doyensys proactively monitors Oracle Enterprise Manager 13c. The team also uses an apex custom-developed tool to monitor the environment. Moreover, there is an administration team that performs three scheduled checkups of databases daily.
During this process, the team checks for system and database log messages and verifies the space usage. Additionally, the team members also check for long-running sessions and inspect the database system status.
The database team also generates and analyzes regular database AWR reports (Report generation is automated on all PROD environment). If they find any poor-performing SQL statements, they co-ordinate with other teams to optimize the SQL statements to enhance the query performance.
EBS Architecture
The client’s financial ERP system has a three-tier architecture, which includes two for web and one for concurrent. An external load balancing tool shares the load across the node in a round-robin method. SSL is implemented at the load balancer level, which adds security to the system. Apart from authenticating every user IDCS, there is appgate SSO, which is also a two-step verification process.
Maintenance and Patching
Doyensys has successfully patched one time along with one upgrade from Linux 6.6 to 7.8. The team was able to perform patching on a scheduled basis every quarter.
How Doyensys added value to the client?
After understanding the problem statement and evaluating the system, Doyensys also provided a few enhancements to offer a complete package to the client.
For instance, some of the customer’s EBS instances are refreshed every Tuesday and Saturday. The Doyensys team automated this process to almost zero manual intervention. The custom shell script triggers itself to clone the database, scrub the sensitive production data, create custom database links, reset passwords, tar and untar application tier, clone the application tier, etc. are all done by itself.
Secondly, the team scripted everything in the environment. It included creating a standby database, switchover, failover, cloning, monitoring, code deployment, etc. The monitoring tool DBMON is getting better every day as the team keeps on updating and writing the new shell script to monitor the environment. Earlier it was just a core database, but now it is capable of monitoring Oracle 12.2 EBS Applications and a few areas of Weblogics as well.
In Conclusion
Working with this client and on this project allowed Doyensys to demonstrate its capabilities in yet another area. We are proud of delivering on this project and exceeding the client’s expectations.
