Straight Through Processing was defined and potential solutions were introduced in the November/December 1999 issue of ABA Trust & Investments. This second article on STP reviews the infrastructure changes the industry intends to implement to achieve STP and several of the techniques that are available to help firms during this transition.

STP was defined as customer to customer automation. This definition is based upon the belief that every important financial transaction begins with a customer need and ends with the customer being notified, updated or otherwise informed. Ideally, the customer should initiate these transactions electronically, the closure with the customer should be electronic, and everything in between should be automated.

This is certainly not the case today. In the past, financial firms automated islands of activity and built numerous connections between these transactional silos. While information can move from activity to activity, the movement was not designed to be seamless, and significant exception processing is still required.

In the next few years, the securities industry will undergo significant change as settlement periods continue to be shortened and the infrastructure looks for new ways to improve STP.

• The US will probably (although not definitely) reduce the regular settlement period from three days to one day in mid-2002. This shorter settlement period will require a significant reengineering of the process flow between the investment managers, brokers, banks and the infrastructure as the process shifts substantially away from batch processing towards more real-time processing.
• Cross-border settlement today is plagued by fails and manual processes. The implementation of the Global Straight Through Processing Association's Transaction Flow Manager will require many new interfaces and will radically change the way that cross-border business is conducted.

These changes will make many current applications obsolete, and will create new business opportunities for many vendors. While these activities are being defined and ultimately implemented, each firm will have to look for ways to connect to other firms in the industry as well as the infrastructure, and to improve their internal process.

This article is divided into three parts:

• Internal STP Solutions
• External STP Solutions
• STP Techniques

INTERNAL STP SOLUTIONS

To have effective STP, firms must automate internally and connect externally. Internal STP requires completing the automation of the processing steps and connectivity between applications.

AUTOMATION

Over the last thirty years, operations and systems professionals have automated most securities processing functions. Gaps still remain in firm-to-firm communications, but an increase in the use of standards is reducing the communication barriers, and the overall effectiveness of telecommunications technology is improving.

However, many areas within firms have been historically difficult to automate. Sometimes there isn't sufficient volume to justify the cost of automation; sometimes the rules have seemed too complex to specify; and sometimes the effort required to conceptualize the entire change has appeared to be overwhelming. Because of this, there are still several areas in most firms that contain unpleasant, error prone, manual tasks, such as:

Delivery Instructions

A major gap in the industry's drive for Straight Through Processing has been in the area of automated delivery instructions. This process was mostly manual until Thomson developed ALERT, which captured a significant share of the international market. The Depository Trust Company (now renamed the Depository Trust Clearing Corporation) responded by developing the Standing Instruction Database (SID).

Today, four factors are important for the successful introduction of an automated solution for delivery instructions:

• Achieving critical mass by collecting the data from all of the affected firms
• Ensuring reliable connectivity
• Establishing multiple, easy ways to access the data
• Maintaining the data

Significant work remains to be done in the area of message content for many instruments and in the overall delivery timeliness. Content will improve over time as additional instruments are evaluated and added; however, the biggest problem that remains is in maintaining the accuracy and timeliness of the data. All too often the data in the database is out of date, and although the settlement instruction is enriched automatically, the transaction results in an error.

Investigations

Investigations arrive on paper, by voice and electronically, and most are tracked with some form of electronic logging system. However, the actual investigation process for securities is generally manual. Since most investigations tend to fall into a few generic categories, a significant amount of the investigative process can be automated. At the very least, an electronic folder of relevant information can be assembled automatically for the investigator before they even see the customer's question.

For example, errors in posting dividends and interest are usually the result of differences related to positions or rates. When a customer initiates an investigation, the electronic system could assemble information from the securities master file and the customer's transaction history file to ascertain what should have been done and compare it to what was actually processed. With this information assembled before the investigator begins to work, the amount of time saved can be considerable.

This electronic folder can be assembled using a variety of workflow tools, such as those discussed later in this article.

Exception Processing

It has always been easier to develop processing systems that address the routine flow than to automate the exception processing. Typically, firms have concluded that if they try to identify all of the exceptions and automate them in the first release of a new application, the new system may never get completed. So, the usual process is to automate the high-volume, repetitious functions first and come back to exceptions later.

It is also typical that each exception has such a low volume that it may not be economically appropriate to enhance the processing system to accommodate each individual exception. So, once a system has gone live, it becomes difficult to justify the enhancements that automate specific exceptions.

Since users and systems developers both know that enhancements are difficult to justify, they often debate what should be included in the first release. The systems staff would prefer to keep the application as simple as possible so that they can deliver it faster and so that it has a greater probability of working correctly. Users try to get in as many functions as possible in the initial release because they don't know if the budget will still be there in the future, or if some other project will take priority over their enhancements.

This conflict is typically complicated by the fact that exceptions often result from rapidly changing or evolving rules. Defining these rules is difficult for the users, and hard-coding these changing situations is impractical. However, with some of today's workflow tools, it is possible to establish a table of rules that can be maintained by users as these rules change, and to embed these rules into the process flow itself.

Exception processing is usually found in areas such as transaction initiation, resolution of fails, corporate action processing, etc.

CONNECTIVITY

In order to establish internal interfaces between existing applications within firms, there are several issues that must be resolved.

Data Availability

While it seems axiomatic that data must be captured electronically in order to process transactions electronically, many firms are unable to complete their move towards internal STP because their systems were not initially designed to identify and hold the data elements that are required today.

Adding data elements to the files of legacy systems is often difficult and firms may build additional databases for specific new data elements.

Timing

When the data is available, it must be available in time to be processed. Firms are increasingly unable to rely on their legacy batch systems to provide data that is needed immediately for processing. For example, to implement STP in a T+1 environment, firms need information about securities that have not been previously traded. To meet this need in today's T+3 environment, most firms establish a dummy securities number and update their systems in batch mode at night. In the future, when the transaction is supposed to settle the same night as the trade, the required information will have to be acquired on-line in real-time or in near real-time.

Real time processing is expensive! Most IT departments are constantly trying to balance the need for immediate processing with the need to control costs. Increasingly, firms will have to replace their batch transaction processing systems with real-time applications. Some reporting activities and functions such as corporate actions and income processing may possibly remain in batch.

Technology

An IT cliché is that any automation can be accomplished given enough time and money. Unfortunately, most firms find both of these resources in short supply. That means that each firm will have to decide whether it will continue to rely on its existing technology, begin a long and painful migration towards newer, more efficient technologies, or outsource IT development and maintenance.

Of course, the key question is which technology to adopt. Five years ago, the conventional wisdom was that Windows NT would soon replace mainframe and UNIX operating systems before the end of the decade. This hasn't happened since UNIX systems have proven their reliability and the demise of the mainframe has been greatly exaggerated.

When the data is available, firms historically connected applications by writing specific programs to move the selected data. In recent years much of this connectivity has been accomplished by a new category of systems tools called middleware, which is discussed in the section of this article on techniques.

EXTERNAL STP SOLUTIONS

Concurrent with resolving their internal STP problems, firms can and should be working on external connectivity solutions. For years, most external connectivity was the result of hard-coded, point to point connections that were established bi-laterally. Increasingly, vendors are working to provide pre-packaged or at least semi-packaged solutions.

The primary processing issues that affect the external Straight Through Processing problem are:

• Formatting
• Connectivity
• Reconcilement

FORMATTING

In a typical external interface, an extract program must be written to select the required information from one or several internal applications, and then the data must be formatted so that the receiving entity can understand the message's contents. This requires that a common message standard be used and that the message contains the necessary information for the next step in the process.

To make messaging work efficiently the industry has accepted the challenge to develop message standards. S.W.I.F.T., ISITC and FIX have supplied, and are continuing to supply, a variety of useful message formats. When both sides to a transaction use the same format, the chances of a misunderstanding are reduced, and the processing is more efficient.

The securities industry is both blessed and cursed with a multitude of standards. The blessing is that there are many standards to choose from and many standards organizations that are all working towards solving industry problems. The curse is that there isn't one single standard. While most processing businesses would prefer to rely on a single standard, that is not yet possible

DTCC, S.W.I.F.T., FIX and Thompson are all attempting to extend their own message domain into other areas of the trade life cycle. However, the different standards that do exist are beginning to migrate in a similar direction, and perhaps one day the industry will have a uniform standard. This new common standard may be based upon the Extensible Markup Language (XML) that has emerged as a result of the Internet.

The information that is collected must be routed to the proper processing channel, and the message must be processed in a protocol that will be recognizable by the required end-point. For instance, if the message should go out via S.W.I.F.T., the router would determine that the message must be routed to the approved S.W.I.F.T. interface device so that the appropriate telecommunications protocol can be used.

Alternatively, if the message was to be sent to the DTCC's ID system, the router would send the information to an interface that would properly connect to the DTCC either through PC Connect, Computer to Computer Facility (CCF) or Mainframe Dual Host (MDH). Each of these services requires a different technical protocol.

The reconcilement of transactions and positions between investment managers and custodian banks is a major area of focus when considering Straight Through Processing improvements. This reconcilement could be on a daily, weekly, or monthly basis.

There are many different ways to reconcile that require an agreement between the investment manager and custodian. These include deciding whether to reconcile balances or transactions, positions or accounting, and what data elements will be included in the match. Each firm also needs to establish various tolerance levels to reduce the amount of manual reconcilement required for small differences.

Once a substantial difference has been identified in the reconcilement process, the overall process of repair should also be automated, and information should be sent to the transaction initiating system to ensure that the firm's records and archives are correct.

There are many tasks in the securities industry that are shared among firms or which require firms to communicate with each other. And, vendors throughout the industry would like to develop products to solve these industry needs. Because of the complexity of these tasks, each vendor normally focuses on a specific niche opportunity and works to connect a subset of firms with a subset of functionality for a subset of instruments. While these vendors can provide very efficient solutions for their specific area of interest, they are not able to provide a comprehensive solution. Realizing that they will never be able to purchase a single solution, customers are beginning to require that vendor's design their products so that they can interface with each other.

Investment managers frequently find themselves at the center of many of these vendor interfaces. An active investment manager must get information from market data vendors, order indication systems, a few order routing systems, electronic crossing networks, confirmation systems, and an interface with at least one custody system. To manage this information flow, investment managers either buy or build trading systems to manage their workflow, as well as portfolio management systems that are used for their primary reporting and analysis.

Brokers find themselves in an even more complex situation because they have to interface with multiple vendors in each classification in order to support their need to support multiple investment manager clients. Rather than deal with one order indication system, for instance, a broker may find itself with interfaces to all of the available order indications systems, all of the order routing systems, and all of the confirmation systems.

STP TECHNIQUES

Outsourcing has become an essential strategy for forward-thinking IS managers. It is no longer just a way to reduce costs, but is now a critical tool that can be used to add value to a business.

The correct application of outsourcing allows organizations to concentrate on their core business, while assigning systems maintenance and development tasks to firms that are specifically designed to deliver this service efficiently and with a high degree of quality. Outsourcing clients are able to redeploy their own critical and scarce resources towards developing their product and their market share, and are no longer concerned about hiring, training and retaining technical resources.

Brokers increasingly are using correspondent clearers to eliminate their own need to develop and maintain processing systems that are no longer effective in differentiating them from their competitors. Most brokers focus on providing an effective interface with their customers, whether that interface is through a registered representative using a telephone, or the Internet.

Firms throughout the industry are considering additional forms of outsourcing. These solutions range from providing service bureau processing for a single function to outsourcing of all of the firm's operations and systems.

Outsourcing will expand significantly in the US when the settlement cycle is shortened.

In recent years, there has been an increased focus on the concept of moving messages from application to application rather than moving files. This increase in the use of messaging has driven the need for flexible applications that can reformat and reroute these messages so that they can be easily read and understood by the appropriate systems.

Messaging involves individual transactions that are sent from point to point as they occur. The message is a single transaction that is sent by itself to another application for further processing, while a file typically contains many transactions.

Messaging is also associated with real-time processing, where the actual processing occurs as soon as the transaction is received. The opposite of real-time processing is batch processing, where a file of transactions is held until the system is ready to process them. While the industry trend is towards increased use of real-time processing (or near real-time) and messaging, there is still a definite place for batch processing to support operations.

It is clear that an order for a trade must be sent and processed quickly, and therefore messaging and real-time processing are appropriate. It is also clear that in today's environment, we can easily process transactions such as dividends in an overnight batch, so not every transaction needs to be converted to real-time processing.

Most firms have a technology architecture that evolved as users defined their changing needs, specific applications were acquired and as new technologies were developed. Efficient and high quality processing increasingly requires the movement of data in the form of a message between these legacy applications, new applications, and to/from other firms and industry utilities. This increase in the use of messaging has increased the need for flexible applications that can reformat and reroute these messages so that they can be read by the appropriate systems.

In the past, programmers wrote specific code that generated the application's output in a specified format and told the network where to send it. This worked fine when there were only a few applications and a few end points; however, as the number of both has increased, it has become more and more difficult to maintain the hard coded instructions, and the need for a flexible application increased. The new category of application is called Middleware.

The Middleware layer of code sits between the processing applications and the network and performs three basic functions: reformatting, routing and protocol connectivity. Middleware eliminates the need to hard code these connections between applications or firms.

figure 1. Middleware Domain

In addition to routing and formatting, systems developers have had to contend with various telecommunications protocols, which describe the technical points of connectivity. To provide true Middleware functionality, an application must also support the transfer of a message from one protocol to another while routing and formatting.

The use of Middleware will continue to expand as real-time processing and messaging become more prevalent, as developers realize that it is not economical to hard code every interface between applications and firms, and as the number of different points of connectivity continues to increase.

The reengineering pendulum has finally begun swinging back to where operations managers have known it should have been all along. Operations managers have known that their primary job has always been to maintain a high level of control and quality while continuously improving efficiency. However, in the first half of this decade, this focus was confused as multiple consultants and academics promoted rapid downsizing and business reengineering.

Early in this decade, senior managers were too quick to apply the techniques behind reengineering and the consultants and academics were too busy delivering services to notice that firms were misapplying the concepts and damaging themselves in the process. The academic approach that is based upon complex teams and massive reorganizations was appealing to senior managers because it promised a panacea - and quickly. Instead, firms created a demoralized, fragmented workforce that was being asked to do more work with the same inadequate tools they always had.

At last, the person who was probably the most responsible for starting the downsizing of the American workforce, Michael Hammer, author of Reengineering the Corporation: A Manifesto for Business Revolution, has recently stated that he realizes that the most important factor in process change is the people involved in the process.

This new understanding is spreading rapidly. "People are starting to realize that changing how people work is more important than reengineering," says Thomas Davenport, a professor at the University of Texas. For operations managers who are already engaged in the task of changing their processes every day that does not come as a surprise.

Firms cannot change their organization or their process without also altering the tools their people use. Throughout the industry, firms have cut layers of management without redefining the survivors' roles and without altering the flow of work. This has led to situations where managers above and below the eliminated layer have had to assume the responsibilities of the manager who was eliminated.

This has led to longer hours, and if that was not possible, some tasks were just dropped. While much of what has been dropped was no longer necessary, the selection of what would be dropped was not usually made as a conscious decision. Individuals and managers often made it as they passed their saturation point.

This in turn has led to increased operational risk and to an environment where managers have begun to feel that they could decide what to do and what to drop. This is one contributing factor to the increased level of control and operational problems that have plagued the industry recently.

The need for reengineering isn't dead, nor is the concept; however, firms have begun to realize that the real objective of reengineering is to increase the profitability of the firm, not merely to downsize. The focus should be as much on increasing revenue as on decreasing costs.

There are two major categories of reengineering. One focuses only on operations and the other considers the entire business.

Process Redesign

As previously stated, the day to day role of the manager is to constantly reevaluate the efficiency of their operation and constantly engage in process redesign. There are a number of very specific tools and techniques that can be used in this level of redesign: all of which revolve around the concepts of Measure, Manage, and Monitor.

Process redesign requires that managers establish some form of measurement tools so that they know what they are doing. Knowing volumes, timeliness, error rates, sources or errors, etc., is essential to defining possible solutions and improvements. Once a new process is implemented, the manager must continue to monitor the process to verify that the situation actually improves. If not, try another adjustment.

Business Redesign

Reengineering at the business level is more complex, and has usually been less successful since it involves multiple disciplines, and an in-depth understanding of the trends that are driving the industry, what customers want, and how the competition is preparing to deal with the future. Business Redesign requires firms to make major changes to their assumptions and to their habits. These are the areas that Hammer and Champy were really driving towards - a fundamental and radical change in the way businesses behave and in their role with their customers - not merely cutting operational costs.

Several of the business trends that are affecting the securities industry were identified in the previous article on STP.

Document imaging is the process of taking a paper-based document and transforming it into a digital image for storage and retrieval. This technology has become an integral part of the way organizations conduct their business. It has developed from a simple storage and retrieval application into the electronic management of images and associated text files, which is called document image management.

The systems themselves have evolved from standalone, proprietary systems to image management software that runs on industry standard platforms, typically using a client/server architecture with industry standard personal computers connected via a LAN to the imaging workstations. Many firms in the securities industry, such as Northern Trust and recently TIAA-CREF, have successfully employed this technique.

The types of processing stages that a document must go through when being converted from paper to digital format vary depending on the way that users will access the document once digitized, and on whether or not the firm wants to add Optical Character Recognition to the workflow. However, a typical image processing system consists of four phases:

• The Input Phase includes the procedures required to get the image information into the program, such as faxing, scanning or importing image files.
• The Processing Phase encompasses the operations that the document image managers perform to process the image data into a suitable format for later retrieval and orchestration.
• The Access Phase provides search, retrieval, navigation and other manipulative tools.
• The Output Phase encompasses distribution of images, either by paper or electronic means. Many prospective users own neither the financial nor the organizational resources to implement imaging across the entire enterprise in a single stroke.

With these constraints, it's far easier to justify the application of new technology to an existing, bounded business processing environment than it is to populate every desktop. Therefore, most banks adopt the same strategy: start small and then expand in phases.

Systems have been created over the past thirty years to solve individual department solutions, and very few institutions have a firm-wide architecture in place. It usually falls to the operations staff to work with these multiple applications, often on different platforms, and provide the glue that makes the overall process work.

It is possible using today's tools (PC's LAN's, Internet and client/server techniques) to link these disparate applications together by using another layer of technology - workflow. With a properly designed workflow application, the firm can capture the image of any document that cannot be eliminated, and add information to it so that it can be automatically processed.

The workflow application can be designed to post information to another application, obtain information from another application, or to provide the existing data in a screen format for some form of human intervention. After all of the required transactional information has been collected, the workflow system can then forward the completed transaction to the next step in the process, the next department, or an external interface.

It is also possible to establish the workflow process with a series of embedded rules. When embedded rules are used within an automated workflow process, the rules table can be accessed to determine what information should be added, who needs to be contacted for approvals, and where the file transaction should be routed.

Most importantly, with workflow software such as Diamonds, provided by EPM Software, the rules can generally be changed by the users under some management control as the processing circumstances change. This saves time and expense.

A constant internal processing problem concerns storing information and archiving. Financial services firms are increasingly aware that the electronic storage and retrieval of data is critical to their success, but storing records on paper is not practical, and while storing on microfiche is inexpensive, it is expensive to recover. In making storage decisions, firms need to consider the cost and frequency of retrieval as much as they consider the cost of the storage itself.

Many alternatives exist. Each method has strengths, weaknesses and specific costs. Although the data center is measured on its ability to manage the overall storage costs, the real cost to the firm is calculated in the time it takes to recover the data in a usable form. The fastest and easiest method of recovery for the user is from on-line, hard disk storage. This is also the most expensive.

Figure 4. Comparison of Cost Categories

However, the cost of data storage is decreasing. Five years ago, hard-drive data storage cost about $1.00 per megabyte; today, the cost per megabyte is measured in fractions of pennies. This does not necessarily say that all data should be maintained on hard-drives, because there is also a people-cost involved in adding and maintaining new drives.

One concept that is being used to balance the need for cost containment vs. people-time is called Hierarchical Storage Management. HSM suggests that firms free up on-line hard-disk drives by migrating infrequently accessed files to another medium - typically near-line optical or tape storage. Some of the tools available for this storage are RAID cabinets, tape loaders, optical libraries, and CD-ROM jukeboxes. While the data center staff will probably make the final determination of which type of hardware to use, operations professionals should be aware of the implications.

Data Warehousing

Data warehousing refers to the strategic use of data, and not to the actual means of storage. It usually consists of replicated data, and often can be accessed by people throughout the firm and possibly even by customers.

While some operationally oriented data could ultimately reside in a data warehouse, the operations function itself is probably not interested in using this accumulated data. Operations staff are more concerned with the ability to store operationally oriented data inexpensively and recover it quickly and easily when needed.

Recovering Archived Data

Most firms have a process that routinely stores key data in archives, on and off-site. This is an important audit requirement, and is also good business. One factor that is often overlooked is the firm's ability to recover and use the data. It is important to remember that whenever file structures are changed significantly, or applications enhanced, it is possible that the new application will not be able to read the old data.

SUMMARY

In summary, the key components of Straight Through Processing for internal connectivity include automation and connectivity. The key components for external connectivity include formatting, connectivity, and reconcilement.

There are several techniques available to assist firms in preparing for STP, including outsourcing, middleware, reengineering, imaging, workflow, and electronic data storage and retrieval.

Vendors and industry utilities are attempting to meet the demand for new products and services, and standards organizations are attempting to work together to simplify the way that the industry communicates.