Enterprise Application Design
One thing we have learned over the years is that you should always design an application so that it is possible to scale it. Even though a particular application may not be intended for use by more than a handful of people from the start, many such applications have a tendency to grow in popularity, and suddenly hundreds of users want it. If you’ve designed with scalability in mind, from the beginning, you really won’t have a problem when this happens. By simply separating the different layers from each other and then placing them on different computers or clusters, you suddenly have an application that can serve many users without a complete rewrite.
In the Beginning . . . Back in the eighties, stock traders on Wall Street used stand-alone terminals. But it soon became obvious that they needed a way to link the terminals, asset trading data, and management systems together so that it would be easier to get an overview of all brokerage information, allowing them to make better decisions. This may have been when the concept of application integration was born. After that, many other businesses found they had similar needs.
Many things have pushed the need for integration to the forefront of business concerns. Client-server solutions started taking the place of mainframe systems, for instance, and developers were suddenly able to build more flexible solutions for their companies. In those days, many of the solutions and applications were proprietary to the companies that had developed them. When two companies merged, heterogeneous environments made it difficult to map data. The complexity of all these applications needed to be reduced. A way to get these applications to work together was required, and the answer was application integration.
With the Internet explosion in the nineties, new ways for companies to do their business evolved. Business-to-business (B2B) and business-to-consumer (B2C) opportunities unavoidably led to a need for integration, not only within companies, but also between companies and their customers.
The first generation of EAI solutions was often a hub-and-spoke architecture (where a single integration server, the hub, handles the information exchange and transformation for the spokes, or many applications or data stores) or of an integration bus-oriented one. Suddenly, a logical separation started to appear in applications. Data was separated from transport, requests from responses, publisher from subscriber, and so on. Slowly, standards like CORBA and COM started to introduce themselves. People began talking about loosely coupled communication. A problem with CORBA and COM, however, was that they were still fairly proprietary. It was not easy to integrate these two standards. There were also some performance and scalability problems with these methods.
The constant evolution of business continuously drives a change to EAI. Nowadays the term has expanded to also include message brokering, business process flow management, and workflow. As companies try to reduce costs, it becomes more and more important to simplify the creation, management, and modification of integrated applications. Companies can’t rely on technicians and developers all the time when a change has to be made in the IT infrastructure or in the business processes. There is also a great need to reuse business logic to cut costs and reduce complexity. Fortunately, standards like XML, SOAP, web services, and others have been developed that make it easier, more cost effective, and safer to integrate.
Let’s take a look at an imaginary company that manufactures and sells cars, called R & R Automobile Corporation. R & R has been around for quite a number of years now. During this time, hundreds of data and communications systems have grown up in the company, and so many, in fact, that nobody knows the exact number. These systems reside on different hardware and software platforms: 70 percent of these probably reside on mainframes, while the rest are scattered on client-server and PC systems. Data is transferred between these systems at different times. Some systems transfer or receive data in real-time, while others transfer or receive data in batches, on a daily, or even weekly, basis. A few systems cannot communicate with others at all, and R & R staff has to print information from those systems and manually enter it into other systems.
R & R management has discovered the need for a closer interaction with both partners and customers. This means that the supply chain, customer support, and service need to be tuned so that R & R can be an effective, modern company, which management believes can be achieved by providing a better integration between the applications. Obstacles have arisen in the way of achieving this, however. Through all these years that R & R has existed as a company, boundaries have been established between units, including geographically dispersed offices, product lines, and so on, and these factors all diminish the ability of employees to share information within the company. As you might understand, it’s hard to give value to the customer if the obstacles within the company are as big as this. Nevertheless, integration across the enterprise probably will provide significant opportunities to show a unified front to the customer. It will also most likely supply more efficient end-to-end processes.
Take a look at some of the problems, or perhaps challenges, that face an R & R integration team:
- R & R uses a mix of technologies. Over the years various technologies have been used to build R & R’s applications and databases. As mentioned, some systems provide an interface to the surrounding world, while others do not. The company is rife with customer relationship management (CRM) systems, and other applications, both standard and proprietary. These applications are often run on different platforms.
- New applications at R & R need to extend the features provided with legacy systems. The cost of updating the legacy systems is too great to be considered as a serious alternative, and some of the systems just can’t be changed.
- New business workflows need to be developed. R & R needs to find ways to develop new workflows for its business processes, while somehow incorporating its existing applications.
- Many of the applications in use at R & R have no clear separation between their business logic, user interface, and data access. In a modern n-tier application, designers and developers architect the application to separate these items, as shown in Figure 1-2.
- The data transferred between applications is provided in various formats. EDI documents are used, but so are text files and XML files. It is therefore difficult to map the fields from one data structure to the fields in another.
- Different component models are used. Some applications use CORBA (Common Object Request Broker Architecture) and others use COM/COM+. This has resulted in tightly coupled applications that do not easily communicate with each other.
- R & R’s systems are not integrated with the outside world, and the issues so far have been within the company. But, at a moment when R & R is about to interact with its customers and partners on the outside, most of these issues remain. An additional problem that R & R must face is that it has no control over these other systems. This means the R & R team has to adjust to the ways the outside world communicates.
- Most of R &R’s legacy systems run smoothly. Management knows that changing a legacy system to provide new features involves the risk of introducing bugs and errors. Is it worth the time and money it takes to do such changes? Also, in order to implement any changes, the integration team also needs documentation that often doesn’t exist anymore. If this documentation is found, it will take quite some time for the developers to go through it so that they can understand the systems.
- Any new integration solution deployed today must also be open, so that future infrastructure requirements can be supported. To comply with this, a lot of thought has to go into the design of the solution. R & R will also have to carefully consider the technologies the company is using, so that they are based on industry standards as much as possible. R & R cannot afford to fall for the latest hype when choosing a technology, at least not if that technology is relatively unknown or unproven.
As you clearly can see, integration will not be an easy task for the team at R & R. You can actually think of this as a puzzle. All the pieces are scattered around on the floor. The work ahead of the team is to build a complete picture from these pieces. This is the vision you should strive for in your own integration attempts. As with all visions, you will probably never reach perfection, but if your aim is high, you may come pretty close. And fortunately, there are some tools and techniques available that can help get you on your way.
Before starting any new development project, take a moment (preferably more than a moment, to be honest, especially if you want your project to succeed) and consider a few things. First of all, you should know which problem or problems you really need to solve. Then, you should consider what parts of the development will give the most value to the company. After that, you can start identifying what you need to build and what you need to connect. You also need to be sure to have solicited feedback from management (or whoever is paying the bill) as well, since meeting their expectations is crucial to the success of the project. You don’t want to build something that your employers don’t want. If you do, it doesn’t matter how great an application or system you have built, it will still be a failure.
These points might seem obvious to you, but in our experience we have found it valuable to get answers to all of these questions on paper, so that we know everybody is working towards the same goal.