Sunday, January 28, 2007
(Selling More, By Having An Adaptive System, An XYZ Company Story)
Section 6) Developing Enterprise Systems - Some Guidelines
Section Contents
1 Overview
2 Architectural Goals for Enterprise Level Systems
2.1 High Level Concepts
2.1.1 THE BASICS - Blueprint, Framework, and Standards
2.1.2 Business Domain\Functional Area Database Model
2.2 Patterns
2.2.1 Modular Architectural Pattern\Business Domain Modeling
2.2.2 Layers Architectural Pattern
2.2.3 Software\Business Patterns In General
2.3 Techniques
2.3.1 Components Versus Objects – A System’s Building Blocks
2.3.2 To Components And Beyond -- Interfaces
2.3.3 Configurable Systems Through Use Of Meta-Dat
2.3.4 Code Generation
2.4 Being Forward Looking
2.4.1 Scalability Versus Performance
2.4.2 Technology Usage
2.4.3 Long System Life
2.5 Development Methodology
2.5.1 Iterative Development With Regular Review
2.5.2 Test Driven Programming and Continuous Integration
2.6 Low Level Concepts
2.6.1 Flexible Deployment
2.6.2 Provide Instrumentation Of Process And Data Pathways
2.6.3 Components - Further Considerations
2.6.4 Simplify Format Of Data Storage And Manipulation
4 Conclusion
5 References
1 Overview
This document provides a short selection of important topics, which should be considered by an architect of an enterprise system. Additional information about managing enterprise system development can be found in the list of excellent references provided at the end of this document.
This document looks at things primarily from a development\architectural perspective, rather then an overall project management view. Thus, many other important things that contribute to the success of any enterprise system development effort, such as having strong sponsors, active participation by managers and key business people, are not discussed here.
2 Architectural Goals for Enterprise Level Systems
The table of contents of this document identifies the list of architectural\design issues that I discuss below, all of which I consider attributes of adaptable enterprise systems.
2.1 High Level Concepts
This section of the document presents some high level concepts that should be definitely followed.
2.1.1 THE BASICS - Blueprint, Framework, and Standards
Large software systems should be implemented according to an overall guiding blueprint. A practice that is present in most other engineering disciplines developing any kind of large or complex system.
Can you imagine trying to build a large office building without a blueprint? Who would be able to review or approve the proposed design? How would other engineers (plumbers, electricians, and telecommunication experts) coordinate their activity so they didn’t end up working at odds with each other? How would they know if their proposed effort will be adequate to support other trades people efforts?
With large enterprise systems that can easily end up with a million plus lines of code, everyone needs to be working with the same overall plan if good re-use is to be made of each other’s efforts. Without a guiding blueprint, components and services designed by different individuals will not necessarily work together well, if at all. Following a good blueprint can also help developers build components and services within a consistent vision of how the entire system needs to scale to handle future growth.
Furthermore, all development work should be carried out according to agreed upon standards, which cover conventions for naming, coding, formatting, and organization of code units into modules.
The company’s development activity should have a fulltime hands on software architect and technical lead to coordinate the use of standards, the blueprint, and framework.
What use are standards?
If individual items within a large system that perform similar activities are consistently named, people can guess what something does just by its name. Naming can also be used to organize all the stored procedures or tables that cover one business domain together, making it easier to identify and work on that functional area.
2.1.2 Business Domain\Functional Area Database Models
In regards to design, the next level of detail that enterprise system development should proceed to, is the development of business domain\functional area database models with the goal of producing a common language and understanding between business and IT personnel for those business activities. And, subsequently offer guidance for the actual development.
Divide and conquer should be the mantra for any large system. Humans are only capable of juggling so much detail and interactions at one time. There’s an often quoted rule that the maximum number of items a person can visualize at one time, is between five and seven. So scoping deliverables at an appropriate level makes it easier to work on large solutions successfully.
2.2 Patterns
Software Patterns offer the opportunity to learn from other people’s successes and avoid mistakes.
2.2.1 Modular Architectural Pattern\Business Domain Modeling
Modularization is one of the fundamental software architectural patterns that’s been successfully applied to build all large enterprise systems over the last twenty years. It involves the break-up of the enterprise system into smaller relatively self contained subsystems that handle major functional areas of the business. For example, in the case of a manufacturing company, suitable modules might be: Product Development\ Merchandising, Ordering, Logistics, and Finance. These smaller subsystems can then be developed and installed separately or even substituted with best of breed applications.
This reduction in the reliance on direct linkages between the different areas of a system is commonly referred to as “loose coupling.” Because of the use of indirect links\data exchanges, modules can for the most part be individually enhanced and tested without risking cascading side affects to other parts of the system. It is also the reason why many vendors of commercial enterprise systems can supply individual companies with paired down versions of their products containing just the modules necessary for running a particular company’s activities. One additional benefit of loose coupling is that it dovetails well with the use of Business Domain\Functional Area Database Models, (core system design documents).
2.2.2 Layers Architectural Pattern
Layering is another one of the fundamental software architectural patterns that’s been successfully applied to build large systems of all sorts. It’s a tried and true technique for packaging cohesive elements of a system in their own layer to provide some core service to the rest of the application and even between applications. If the communication protocols across the boundaries of each layer are respected by all parts of the system, the classes within those layer scan be significantly changed without interfering with the rest of the application as long as each layer continues to service its original communication interfaces.
This is one of the most challenging areas of building enterprise systems, as it requires that people be able to think abstractly about what services need to be made widely available, and how they might be generically implemented to facilitate their reuse. And which services are similar enough in nature, that they should be packaged as a larger cohesive part of the system. Often the best guidance on this topic is to review other publicly available frameworks to see what types of things they do, and what parts of them could be adapted for ones own use.
A common use of the layering pattern in enterprise systems is to inoculate the rest of a system from any undue influence of a particular data access technological. Such technologies tend to change every two to three years and systems in which it has not been isolated in a single layer get caught in a situation where there is no easy path to the new data access technology and they end up tending to use the older technology well beyond it best use date.
It’s not necessary that a layer be an actual physical separation within the application as long as the functionality that layer represents is well scoped and does not vary across its implementations. For example, the basic CRUD (Create, Read, update, and Delete) operations of business components could be grouped together within a component and named the same across all business components. When it comes time to update those routines for a new data access technology they can all be easily located and dealt with together. Current features in .NET 2.0 languages such as code folding regions and partial classes provide some nice mechanisms to facilitate such activity.
An excellent use of layering that is not used commonly enough is to put business logic\rules inside business components that live within a business layer that is accessible by the remaining upper layers of the system. This would provide the different presentation layers the same direct access to that business logic, reducing the amount of code that needs to be written within them. For example, this would permit both a Windows and Web application to make use of the same business component to check when a customer’s next delivery can be expected or whether his account balance is in the black.
Layering also facilitates the application of individual developer skills to parts of an enterprise system to which they are best suited. Those that are better with databases can work on the data access layer, those who are better thinking along abstract lines can spend more time working on the core reusable services, and those who have web experience can work on browser base forms.
2.2.3 Software\Business Patterns In General
Many great efforts have been made over the last ten years to document software and business domain patterns that offer solutions which can be reused within other systems. These patterns provide tried and true techniques for solving problems commonly encountered in building systems. They give software architects and developers immediate access to techniques they can used to solve problems they are faced with, without requiring them to go through all the trial and error the original authors did. They also provide the ability to architects and developers to describe how major areas of functionality within the system works, by just naming the underlying patterns with which they were constructed.
Documented patterns can be found for many levels of software development. There are patterns called “design patterns” that solve many smaller problems within an application. Some commonly known design patterns are: singleton, bridge, facade, strategy, and chain of command. There probably isn’t a programmer who has built a large system who hasn’t used all of those patterns whether he was aware of there names are not. Details on when and where to use particular design patterns can be found in the classical reference on the subject, Design Patterns by Gamma et al. There are more substantial patterns that can be used for the overall architecture of an enterprise system. Some commonly used architectural patterns besides the ones mentioned earlier are: pipes and filters, broker, microkernel, and model-view-controller. Additional information on when and where to use these patterns can be found in the book Pattern Oriented Software Architecture – A System Of Patterns by Buschmann et al. There are even collections of patterns for individual business domains such as accounting and finance, observation and measurement, and trading, Martin Fowler’s book Analysis Patterns – Reusable Object Models has more to say on that subject.
Any new enterprise development effort should try and benefit from the knowledge of others in order to deliver a successful solution with the least effort that will withstand the test of time. Patterns offer a quick means of doing that. The one question I asked all developer candidates is, “Have you heard about, or do you know about software patterns,” their answer generally allows me gauge whether they keep up with best practices in the industry and are open to learning from others. At the end of the day, the enterprise development effort is its developers and what they know.
2.3 Techniques
The following sections discuss some very specific things that enterprise system builders should be doing.
2.3.1 Components Versus Objects – A System’s Building Blocks
Components involve the packaging of smaller objects into a container capable of managing them to provide a more robust implementation of their combined functionality. By packaging smaller objects into larger containers capable of handling bigger units of work, other parts of the system (and developers) need know less about the specifics of what the individual objects do, or the details of what it takes to make them work together, making it much easier to re-use them. Actually, even though objects are the smaller elements of a system they too should be constructed with well defined cohesive behaviors with minimal interdependencies so that larger system items such as components can more easily make use of them.
It is much easier to construct entirely new system modules working with components, rather than objects, as the units of work that can be cobbled together are bigger. They also permit the development team to more quickly adapt a system to changes in business needs, as functionality is packaged at a more comprehensible and manageable level.
2.3.2 To Components And Beyond -- Interfaces
As significant as the movement to emphasizing components over objects has been, so has been the movement to emphasizing that people focus on programming to interfaces versus class hierarchies.
In the past, systems that relied heavily on class hierarchies to get there work done were found to be brittle. Changes to one level of the hierarchy would often break classes in many other levels. The primary problem is that class hierarchies tend to create tightly couple items, which flies in the face of advocating loose coupling between parts. Today, system development work tends to focus more on whether a class can perform a particular set of behaviors as identified by a specific interface (contract). Interfaces represent a set of methods with particular signatures that define a collection of behaviors. Any individual class can implement a number of different interfaces to represent a broader range of behaviors, and new interfaces can be added at anytime. Also, as long as an object or component implements a particular interface, it does not matter where in the object hierarchy it is, other parts of the system can make use of it through its interface.
2.3.3 Configurable Systems Through Use Of Meta-Data
Greater emphasis is being placed on meta-data to configure and drive many areas of new systems. In the past, this has been an area that commercial software developers have paid more attention to, then individual organizations have, because of the needs of their packages to be able to accommodate different types of businesses. However, it does also offer individual enterprises the opportunity to do such things as easily change deployment, reporting, and data access options. Or provide the means to produce highly configurable business rules for systems that can be adapted quickly to changing business needs. And probably most importantly of all, it can provide inputs for a code generator capable of producing 50% plus of all code needed by a new enterprise system.
A balance does need to be established between the complexity of the meta-data gathered and its utilization in regards to the size of the development team. If the development team is large enough, it’s easier to have developers dedicated to this activity to produce a sophisticated implementation that provides an excellent return on investment. It’s also easier to provided continuity amongst programmers to keep the solution productive overtime. However, in a smaller programming shop it’s often much harder to get permission to be able to allocate even a single developer part time to such an activity, because each developer has such a wide set of demands to meet already. And even if one does do so successfully, it’s harder to keep productive continuity on this activity overtime, as when an individual programmer leaves such smaller teams, they often take a huge block of unique knowledge with them, (sometimes the entire knowledge about a particular subsystem).
2.3.4 Code Generation
The idea of code generation (code that writes code) has become a hotter topic of late, with many companies having great success using it to cut development costs and delivery schedules, especially, in large new development efforts where they are generating up to 80 percent of the system’s code.
Today’s code generation engines are a better breed then those of years ago. They tend to offer more flexibility through the creation and use of simple templates that instruct the code generator on how to produce all the repetitive and highly structured code that developers would otherwise need to write.
Taking code generation to the next level, where code regeneration can also be productively used to maintain a system as it evolves, is a very challenging task that is normally only achievable if everyone buys into the use of code generation in the first place. However, such achievements are assisted by the current generation of code generators that use templates to provide more flexible output then their predecessors and .NET partial classes which store generated code for a class in separate files to that of its custom written code. Because the templates are parameter driven, small changes can be accommodated by rerunning templates though the code generator with a slightly different set of parameter values and replacing the custom generated code in the partial class files without touching any of the custom code for the class.
2.4 Being Forward Looking
The topics under this section discuss some items that any forward-looking system designer should take into account.
2.4.1 Scalability Versus Performance
The scalability of a system defines how responsive it remains overall while greater work loads (more users\transactions) are placed on it. A well-designed scalable enterprise system is capable of handling hundreds to thousands of people while still being capable of maintaining a good response to each individual.
Generally, enterprise systems are made scaleable by splitting them across tiers, which basically means running parts of the system on different computers dedicated to providing some specialized service or sets of data. While inserting additional tiers does tend to significantly and adversely affect the performance of any single request because of network latency and other issues, the ability to handle additional requests causes little additional system load because of the specialization in what they do. For example, if everyone logging on to your application requires a list of active stores, that information could be gathered from the database by an enterprise tier when the first person makes such a request, and it could then be cached on that tier, so it can be subsequently passed along to all others making the same request. The .NET framework itself also implements pools of shared re-usable resources such as database connections that users can be given access to as needed. When the user is done with the database connection it is returned to the shared pool where it remains ready to be used by the next person needing it. This removes the need for the system to incur the costly activity of creating a database connection every time someone needs one.
An application tier can be run on one or more computers in a load balance matrix (farm) like setup when necessary, to provide scalability to thousands of users. Such a setup can also provide fault tolerance for system, as the loss of any one instance of a tier does not stop the overall system from working.
The Modularization and Layering architectural patterns mentioned above are used to package tiers for system scalability.
2.4.2 Technology Usage
In building an enterprise level application, people should always be looking to the future, not only in regards to where the company might be going, but also as to the likely direction technology might take. Older technology offers less functionality, and is harder and more costly to maintain as time progresses.
Working with current technology makes its easier to find developers, free code, and productivity tools in a larger more active development community.
For example, if one was using the current technology of C# 2.0 and .NET 2.0 one would have access to freely available frameworks that would take care of much of the plumbing of any new system, a major cost and time saver. And the execution of smart business components that are easily re-used across Windows and Web Applications, and Window Services.
2.4.3 Long System Life
Systems always seem to live longer then most people expect. In fact, prior to focusing on being an enterprise developer, I completed a number of departmental level applications that were only expected to be run for six months, where just about all of them ended up lasting 3 to 5+ years. Had I actually programmed against the original estimate of their useful life, the client would have incurred additional cost to extend their life. However, the cost of programming for a much longer life was so small while writing the application, that I did it, which in the end turned out to be a major time savings, as opposed to me having to go back through another design, analysis, and testing cycle six months down the road, when I was no longer as familiar with the original requirements.
It’s even more important to plan for long life with a custom enterprise system built for a growing company, as the costs of retrofitting it later to extend its life is quite costly. Just as with small systems, its durability should be built with a long-term horizon in mind, not just a company’s present needs. It should be many, many years, before a business starts to stress the capacity of a newly developed system. It certainly shouldn’t be necessary to make apologies for a system’s shortcomings shortly after it’s been delivered, (or even as soon as it’s delivered).
2.5 Development Methodology
2.5.1 Iterative Development With Regular Reviews
Completing a detailed enterprise system design up front is an impossible task. Don’t let anyone tell you otherwise. Commonly, when interviewing business people, things are not always recorded correctly, or even interpreted properly. Not to mention, that people forget to tell you about important things because their not part of their everyday activity, or because they assume you know such information already. And also because they can’t easily visualize how a new improved process might be put together on a computer, (they are tool users not builders). In addition, as volumes of detailed information are gathered on a large system, designers tend to lose their ability to keep everything straight in their heads, and plan the perfect design. And its not uncommon for designs put together over a long period to be obsolete before they are implemented as the organization continues to adapt to changing business needs during the design phase.
The only really practical way to develop a large system efficiently is by gathering some high level requirements initially, and then moving on quickly to focus on individual functional areas\modules in an iterative development-cycle, soliciting regular feedback and guidance from actual business people so that regular adjustments can be made in a timely manner while the cost of doing them is still small. With regular feedback cycles and releases of working modules the final system is more likely to have what business users need.
2.5.2 Test Driven Programming and Continuous Integration
Test Driven Programming is a popular technique for ensuring the delivery of robust systems and maintaining them as such. It involves the construction of tests at the same time requirements are specified, where those tests are coded before the business functionality itself. As working code starts passing the tests, one gets assurance that the new functionality meets requirements. In addition, those tests can be executed in a testing harness with automated executions, which allow them to be run at anytime during the initial development process or any subsequent enhancement cycle to make sure that all versions of the system continue to pass all tests.
A complementary activity to Test Driven Programming is the concept of Continuous Integration, which set-ups automatic builds of systems from frequently checked in code, upon which tests are automatically run, and after which a final exception report is emailed to interested parties.
The loose coupling of applications and components described elsewhere in this document permits simpler tests to be created as fewer interdependencies need to be handled.
Some advocates of Test Drive Programming even view the tests as a means of providing some additional documentation as to what a system’s business rules are. However, I personally feel that such information would not be widely used and tests should instead likely make reference to particular rules in some better and more widely accessible classification of business rules.
Test Driven Programming and Continuous Integration can provide pragmatic tools for maintaining a robust system. Vendor tool support and company usage of these techniques is growing strongly.
2.6 Low Level Concepts
2.6.1 Flexible Deployment
The ability to deploy the system in a single tier (on a single PC) or in multiple tiers (individual user PCs plus one or more application, database, or web servers) should be easily configurable from an application’s file settings.
For example, if a remote sales office only has one user, then it might make best sense to run the module(s) he needs on a single PC. Other larger remote offices or facilities might need to run their modules across individual PCs, and at least one server for better performance. The main office with hundreds or thousands of employees may want to make use of separate servers for individual tiers to get good performance for everyone. All of this should be configurable from an easily modified application settings file.
It would also be highly desirable that the application itself be deployable over the web with a single action on the part of the user.
2.6.2 Provide Instrumentation Of Process And Data Pathways.
Using a single framework (or bus architecture) though-out a system provides the opportunity to provide instrumentation of individual processes and data pathways to identify who used what, when, how, and where. Such activity can help in trouble shooting performance problems and in identifying what parts of a system are no longer being used and can be retired
2.6.3 Components - Further Considerations
Some additional considerations that should be made when constructing components are described below.
Components – Promotion Of Abstraction In Design
Promoting abstraction while designing components can make them more flexible. If components are not unduly specific to any current set of steps or datum, they can be more easily enhanced as business needs change.
Components - Design For Behavior Not Data
Many developers new to Object Oriented Programming (OOP) tend to make the rookie mistake of creating too many simple classes that are largely only record set representations of all the tables in their database, a behavior that is encouraged by vendors of object-mapping-relation products, which will automate the creation of such objects. The biggest problem with having a data-centric view in performing OOP is that it tends to result in light weight objects that contain little intelligence. Which requires, that the smarts they should have contained be coded elsewhere in the system where its not easily reused, and therefore, it often ends up getting reproduced everywhere those objects are used.
Rather then simple data objects, it is better to focus on building business components that might represent data from a collection of tables and all the behavior that needs to be wrapped around them to perform some business task to promote code reuse. For example, an Order business component in its most simple form should be capable of holding customer summary data, such as their name, phone number and address, and a separate data line for each product they ordered, along with methods (behavior) to check that the customer has a good credit rating and calculate the time it will take to deliver their goods. If all this data and behavior is wrapped up in a single business component, then it is easier for Windows and Web programmers to build ordering tools that reference that single item to allow people place orders from both sources. And while the company needs to incur the cost of building two different types of interfaces they save money by only building one business component that does a good deal of the real business work.
Components - Normalize Behavior
Just as relational databases should focus on storing data in a normalized form (without repeating data groups) the practice of object oriented programming should focus on the normalization of behavior.
Taking our example of the Order component described in the section “Components - Design For Behavior Not Data.” it’s not necessary that the component actually know shipping times for products to a customer if the shipping system can already calculate that information. It would be better that such behavior only be implemented in one place, most likely the shipping system, and that the Order component would request a shipping schedule after passing the shipping system a list of products and their destination. The goal being, that actual behavior gets coded in only one place, and therefore, there is only one place it ever needs to be updated when changes are needed. And while the actual calculation of a shipping date\schedule could be done elsewhere, users of the Order component do not need to be aware of that. They would just ask the Order component directly for the estimated ship date, whether the component calculates that information itself, or passes the request along elsewhere, should not be of concern to anyone using that component.
2.6.4 Simplify Format Of Data Storage And Manipulation
It’s desirable that data be passed around the system in a fairly rudimentary format that is largely independent of any vendor specific data management technology that is likely to change every few years, as it has done for Microsoft and Borland over the last 15 years. This helps reduce an application’s dependence on any software vendor’s more specialized data management technology, making it easier to replace in the future.
For example, large amounts of table like data in .NET components can be kept in simple tabular structures with rows and columns that support the .NET Binding List interface that provides data binding for forms permitting bidirectional data exchanges between both. This can be done in preference to the much more sophisticated data typed xml datasets, which are much more likely to be subject to change in each new release of .NET and eventual replacement within two to three years when the vendor creates a new data access technology. The simpler data structures may change too, but generally those changes will be small and manageable, and possibly not affect the business components at all. Even better, if the behavior around those simple structures is coded in a common ancestor for all business components, then any adjustments that need to be made only need to be adjusted for in one place for all components, which would not be the case for more complex data structures, as they get implemented as independent entities with no common ancestor. Another advantage of the simpler structure is that it’s easier and faster to pass data from one component to another for any additional processing, as the data’s storage container carries much less baggage with it.
3 Conclusion
Building enterprise systems is an inherently complex business. Getting all the requirements and design right at the beginning of a large system never happens. If what was presented in this document didn’t persuade you of that, consider the fact, that no one has the rights to a working crystal ball to predict the future, and stuff happens. So by necessity, system development has to be done while trying to hit a moving target. Change is the only constant in our lives, small changes happen all the time, more fundamental changes occur as time progresses.
In accepting the above facts, this document has shown that one can be proactive about planning to handle change when building large systems. It explained how the development effort will be much easier and ultimately more successful if one selects methodologies, tools, and techniques to craft an adaptable development process that is responsive to changes. Having an adaptive development process, allows the development personnel to make necessary changes while development is underway, while it’s still cheaper to do so, and to produce a system that is better geared to what the business’s needs are at the time of delivery.
Some of the most important guidelines discussed in this document for providing an adaptable system were: 1) Making use of blueprints, domain models, and standards. 2) Adopting proven software patterns. 3) Using coding techniques such as component building, interfaces, meta-data, and code generation. 3) Being forward looking with system scalability, performance tradeoffs, and technology usage 4) Using the Agile methods of Interactive Development, Test Driven Programming, and Continuous Integration. 5) Providing for flexible deployments, instrumentation to monitor system usage, and being careful to adopt a fairly neutral method for data storage and handling of data.
While this document did cover a number of major areas of knowledge that any enterprise software architect\developer should be familiar with, it’s not intended to be a comprehensive list. Overall, software architecture is a pretty large and complex subject area offering many challenges. For those interested in the subject, the References section below provides additional sources of information on it.
In larger companies, the software architectural load can be split amongst architects by allowing them develop specialties from which they can contribute unique knowledge to the overall team effort, while only being required to have a general knowledge about other areas. In smaller companies, the software architect generally needs to cover broader ground in less depth, as the company isn’t often large enough to have someone dedicated to work on that activity full time. In such cases, the architect often also needs to manage the development staff, do project management, and actually help write the software. To compensate for being spread so thin, certain responsibilities need to be pushed further down the line to other development personnel who are willing to accept and learn new skills to help make the enterprise development effort a reality. Skills such as those outlined in this document.
4 References
Analysis Patterns – Reusable Object Models by Martin Fowler, ISBN 0201895420. Provides a catalog of “business domain” patterns.
Applying UML and Patterns—An Introduction to Object-Oriented Analysis and Design by Larman, ISBN 013748807. Provides a foundational discussion for building robust, scalable, maintainable systems, using object technology.
Building Business Intelligence Applications with .NET by Robert Ericsson, ISBN 1584502711. A practical guide to using .NET tools to build applications for OLAP and data mining.
Code Generation in Microsoft .NET by Kathleen Dollard, ISBN 1590591372. Teaches code generation as a scriptable and repeatable process using templates so one is not tied to a particular framework or style. Kathleen is one of the industry leaders and a major advocate of code generation. Her book also has a couple of chapters on code generation for the CSLA framework (refer to the “Expert C# Business Objects” book below).
Component Software – Beyond Object-Oriented Programming by Clemens Szyperski, ISBN 020117885. It’s about developing re-usable off-the-shelf components that handle a significant amount of functionality. It emphasizes that one should not focus on trying to build each new piece of functionality from collections of small independent objects while developing larger systems. One should instead, focus on developing more robust components that wrap up smaller objects into bigger containers that are capable of performing bigger units of work, which in turn, require less effort by other developers to reuse them to build even bigger functional units.
Design Patterns – Elements of Reusable Object-Oriented Software by Erich Gamma, Richard helm, Ralph Johnson, and John Vlissides, ISBN 0210633612. This book is the original classic on software design patterns, and still the best reference on the subject.
Domain Driven Design by Eric Evans ISBN 0321125215. This book makes a persuasive argument for the use of domain models to provide a ubiquitous language that ties the domain experts and technologists together. Also, it emphasizes that really powerful domain models only evolve overtime and that some of the best ideas appear after the initial release of a system.
Enterprise Integration—An Architecture of Enterprise Application Systems Integration by Fred A Cummins, ISBN 0471400106. An excellent book, that reviews the technology landscape defining enterprise integration objectives, and which provides generalized enterprise integration architecture.
Enterprise Integration Patterns—Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe and Bobby Woolf, ISBN 0321200683. This book describes using asynchronous message as a proven strategy for enterprise integration, and offers 65 patterns for building such systems.
Expert C# Business Objects by Rockford Lhotka, ISBN 1590593448. A book about application architecture, design, development, and using object-oriented concepts in .NET. Book describes the building of a “Component-Based, Scaleable, Logical Architecture” (CSLA) in C# .NET, which has been used as the basis of system development efforts in many companies with good success. Also, the company Rocky Lhotka works for relies on his framework and code generation to produce upwards of 80% of the code in new systems they develop for clients.
Object Technology – A Manager’s Guide, 2nd Edition by David A. Taylor, PH.D., ISBN 0210309947. This book is an easily read classic on the subject. One of its central themes is that objects are not about the data they contain, but the abstracted behaviors they represent.
Patterns of Enterprise Application Architecture by Martin Fowler, ISBN 0321127420. A book written by a well known industry expert on the topic of suitable patterns for enterprise architecture. The book is written in two parts, the first part is a short tutorial on developing enterprise applications. The second section and bulk of the book is a detailed reference to patterns that assist in building enterprise applications.
Pattern Oriented Software Architecture—A System of Patterns by Frank Buschmann, Regine Meunier, Hans Sommerland, and Michael Stal. An excellent book concentrating on Architectural Software Patterns such as: Layers, Pipes and Filters, Distributed System Broker, Model-View-Controller, and Microkernel.
Strategic Data-Planning Methodologies by James Martin, ISBN 0138511136. The first good book I ever read on the topic of strategic data planning and still one of my favorites.
Section 6) Developing Enterprise Systems - Some Guidelines
Section Contents
1 Overview
2 Architectural Goals for Enterprise Level Systems
2.1 High Level Concepts
2.1.1 THE BASICS - Blueprint, Framework, and Standards
2.1.2 Business Domain\Functional Area Database Model
2.2 Patterns
2.2.1 Modular Architectural Pattern\Business Domain Modeling
2.2.2 Layers Architectural Pattern
2.2.3 Software\Business Patterns In General
2.3 Techniques
2.3.1 Components Versus Objects – A System’s Building Blocks
2.3.2 To Components And Beyond -- Interfaces
2.3.3 Configurable Systems Through Use Of Meta-Dat
2.3.4 Code Generation
2.4 Being Forward Looking
2.4.1 Scalability Versus Performance
2.4.2 Technology Usage
2.4.3 Long System Life
2.5 Development Methodology
2.5.1 Iterative Development With Regular Review
2.5.2 Test Driven Programming and Continuous Integration
2.6 Low Level Concepts
2.6.1 Flexible Deployment
2.6.2 Provide Instrumentation Of Process And Data Pathways
2.6.3 Components - Further Considerations
2.6.4 Simplify Format Of Data Storage And Manipulation
4 Conclusion
5 References
1 Overview
This document provides a short selection of important topics, which should be considered by an architect of an enterprise system. Additional information about managing enterprise system development can be found in the list of excellent references provided at the end of this document.
This document looks at things primarily from a development\architectural perspective, rather then an overall project management view. Thus, many other important things that contribute to the success of any enterprise system development effort, such as having strong sponsors, active participation by managers and key business people, are not discussed here.
2 Architectural Goals for Enterprise Level Systems
The table of contents of this document identifies the list of architectural\design issues that I discuss below, all of which I consider attributes of adaptable enterprise systems.
2.1 High Level Concepts
This section of the document presents some high level concepts that should be definitely followed.
2.1.1 THE BASICS - Blueprint, Framework, and Standards
Large software systems should be implemented according to an overall guiding blueprint. A practice that is present in most other engineering disciplines developing any kind of large or complex system.
Can you imagine trying to build a large office building without a blueprint? Who would be able to review or approve the proposed design? How would other engineers (plumbers, electricians, and telecommunication experts) coordinate their activity so they didn’t end up working at odds with each other? How would they know if their proposed effort will be adequate to support other trades people efforts?
With large enterprise systems that can easily end up with a million plus lines of code, everyone needs to be working with the same overall plan if good re-use is to be made of each other’s efforts. Without a guiding blueprint, components and services designed by different individuals will not necessarily work together well, if at all. Following a good blueprint can also help developers build components and services within a consistent vision of how the entire system needs to scale to handle future growth.
Furthermore, all development work should be carried out according to agreed upon standards, which cover conventions for naming, coding, formatting, and organization of code units into modules.
The company’s development activity should have a fulltime hands on software architect and technical lead to coordinate the use of standards, the blueprint, and framework.
What use are standards?
If individual items within a large system that perform similar activities are consistently named, people can guess what something does just by its name. Naming can also be used to organize all the stored procedures or tables that cover one business domain together, making it easier to identify and work on that functional area.
2.1.2 Business Domain\Functional Area Database Models
In regards to design, the next level of detail that enterprise system development should proceed to, is the development of business domain\functional area database models with the goal of producing a common language and understanding between business and IT personnel for those business activities. And, subsequently offer guidance for the actual development.
Divide and conquer should be the mantra for any large system. Humans are only capable of juggling so much detail and interactions at one time. There’s an often quoted rule that the maximum number of items a person can visualize at one time, is between five and seven. So scoping deliverables at an appropriate level makes it easier to work on large solutions successfully.
2.2 Patterns
Software Patterns offer the opportunity to learn from other people’s successes and avoid mistakes.
2.2.1 Modular Architectural Pattern\Business Domain Modeling
Modularization is one of the fundamental software architectural patterns that’s been successfully applied to build all large enterprise systems over the last twenty years. It involves the break-up of the enterprise system into smaller relatively self contained subsystems that handle major functional areas of the business. For example, in the case of a manufacturing company, suitable modules might be: Product Development\ Merchandising, Ordering, Logistics, and Finance. These smaller subsystems can then be developed and installed separately or even substituted with best of breed applications.
This reduction in the reliance on direct linkages between the different areas of a system is commonly referred to as “loose coupling.” Because of the use of indirect links\data exchanges, modules can for the most part be individually enhanced and tested without risking cascading side affects to other parts of the system. It is also the reason why many vendors of commercial enterprise systems can supply individual companies with paired down versions of their products containing just the modules necessary for running a particular company’s activities. One additional benefit of loose coupling is that it dovetails well with the use of Business Domain\Functional Area Database Models, (core system design documents).
2.2.2 Layers Architectural Pattern
Layering is another one of the fundamental software architectural patterns that’s been successfully applied to build large systems of all sorts. It’s a tried and true technique for packaging cohesive elements of a system in their own layer to provide some core service to the rest of the application and even between applications. If the communication protocols across the boundaries of each layer are respected by all parts of the system, the classes within those layer scan be significantly changed without interfering with the rest of the application as long as each layer continues to service its original communication interfaces.
This is one of the most challenging areas of building enterprise systems, as it requires that people be able to think abstractly about what services need to be made widely available, and how they might be generically implemented to facilitate their reuse. And which services are similar enough in nature, that they should be packaged as a larger cohesive part of the system. Often the best guidance on this topic is to review other publicly available frameworks to see what types of things they do, and what parts of them could be adapted for ones own use.
A common use of the layering pattern in enterprise systems is to inoculate the rest of a system from any undue influence of a particular data access technological. Such technologies tend to change every two to three years and systems in which it has not been isolated in a single layer get caught in a situation where there is no easy path to the new data access technology and they end up tending to use the older technology well beyond it best use date.
It’s not necessary that a layer be an actual physical separation within the application as long as the functionality that layer represents is well scoped and does not vary across its implementations. For example, the basic CRUD (Create, Read, update, and Delete) operations of business components could be grouped together within a component and named the same across all business components. When it comes time to update those routines for a new data access technology they can all be easily located and dealt with together. Current features in .NET 2.0 languages such as code folding regions and partial classes provide some nice mechanisms to facilitate such activity.
An excellent use of layering that is not used commonly enough is to put business logic\rules inside business components that live within a business layer that is accessible by the remaining upper layers of the system. This would provide the different presentation layers the same direct access to that business logic, reducing the amount of code that needs to be written within them. For example, this would permit both a Windows and Web application to make use of the same business component to check when a customer’s next delivery can be expected or whether his account balance is in the black.
Layering also facilitates the application of individual developer skills to parts of an enterprise system to which they are best suited. Those that are better with databases can work on the data access layer, those who are better thinking along abstract lines can spend more time working on the core reusable services, and those who have web experience can work on browser base forms.
2.2.3 Software\Business Patterns In General
Many great efforts have been made over the last ten years to document software and business domain patterns that offer solutions which can be reused within other systems. These patterns provide tried and true techniques for solving problems commonly encountered in building systems. They give software architects and developers immediate access to techniques they can used to solve problems they are faced with, without requiring them to go through all the trial and error the original authors did. They also provide the ability to architects and developers to describe how major areas of functionality within the system works, by just naming the underlying patterns with which they were constructed.
Documented patterns can be found for many levels of software development. There are patterns called “design patterns” that solve many smaller problems within an application. Some commonly known design patterns are: singleton, bridge, facade, strategy, and chain of command. There probably isn’t a programmer who has built a large system who hasn’t used all of those patterns whether he was aware of there names are not. Details on when and where to use particular design patterns can be found in the classical reference on the subject, Design Patterns by Gamma et al. There are more substantial patterns that can be used for the overall architecture of an enterprise system. Some commonly used architectural patterns besides the ones mentioned earlier are: pipes and filters, broker, microkernel, and model-view-controller. Additional information on when and where to use these patterns can be found in the book Pattern Oriented Software Architecture – A System Of Patterns by Buschmann et al. There are even collections of patterns for individual business domains such as accounting and finance, observation and measurement, and trading, Martin Fowler’s book Analysis Patterns – Reusable Object Models has more to say on that subject.
Any new enterprise development effort should try and benefit from the knowledge of others in order to deliver a successful solution with the least effort that will withstand the test of time. Patterns offer a quick means of doing that. The one question I asked all developer candidates is, “Have you heard about, or do you know about software patterns,” their answer generally allows me gauge whether they keep up with best practices in the industry and are open to learning from others. At the end of the day, the enterprise development effort is its developers and what they know.
2.3 Techniques
The following sections discuss some very specific things that enterprise system builders should be doing.
2.3.1 Components Versus Objects – A System’s Building Blocks
Components involve the packaging of smaller objects into a container capable of managing them to provide a more robust implementation of their combined functionality. By packaging smaller objects into larger containers capable of handling bigger units of work, other parts of the system (and developers) need know less about the specifics of what the individual objects do, or the details of what it takes to make them work together, making it much easier to re-use them. Actually, even though objects are the smaller elements of a system they too should be constructed with well defined cohesive behaviors with minimal interdependencies so that larger system items such as components can more easily make use of them.
It is much easier to construct entirely new system modules working with components, rather than objects, as the units of work that can be cobbled together are bigger. They also permit the development team to more quickly adapt a system to changes in business needs, as functionality is packaged at a more comprehensible and manageable level.
2.3.2 To Components And Beyond -- Interfaces
As significant as the movement to emphasizing components over objects has been, so has been the movement to emphasizing that people focus on programming to interfaces versus class hierarchies.
In the past, systems that relied heavily on class hierarchies to get there work done were found to be brittle. Changes to one level of the hierarchy would often break classes in many other levels. The primary problem is that class hierarchies tend to create tightly couple items, which flies in the face of advocating loose coupling between parts. Today, system development work tends to focus more on whether a class can perform a particular set of behaviors as identified by a specific interface (contract). Interfaces represent a set of methods with particular signatures that define a collection of behaviors. Any individual class can implement a number of different interfaces to represent a broader range of behaviors, and new interfaces can be added at anytime. Also, as long as an object or component implements a particular interface, it does not matter where in the object hierarchy it is, other parts of the system can make use of it through its interface.
2.3.3 Configurable Systems Through Use Of Meta-Data
Greater emphasis is being placed on meta-data to configure and drive many areas of new systems. In the past, this has been an area that commercial software developers have paid more attention to, then individual organizations have, because of the needs of their packages to be able to accommodate different types of businesses. However, it does also offer individual enterprises the opportunity to do such things as easily change deployment, reporting, and data access options. Or provide the means to produce highly configurable business rules for systems that can be adapted quickly to changing business needs. And probably most importantly of all, it can provide inputs for a code generator capable of producing 50% plus of all code needed by a new enterprise system.
A balance does need to be established between the complexity of the meta-data gathered and its utilization in regards to the size of the development team. If the development team is large enough, it’s easier to have developers dedicated to this activity to produce a sophisticated implementation that provides an excellent return on investment. It’s also easier to provided continuity amongst programmers to keep the solution productive overtime. However, in a smaller programming shop it’s often much harder to get permission to be able to allocate even a single developer part time to such an activity, because each developer has such a wide set of demands to meet already. And even if one does do so successfully, it’s harder to keep productive continuity on this activity overtime, as when an individual programmer leaves such smaller teams, they often take a huge block of unique knowledge with them, (sometimes the entire knowledge about a particular subsystem).
2.3.4 Code Generation
The idea of code generation (code that writes code) has become a hotter topic of late, with many companies having great success using it to cut development costs and delivery schedules, especially, in large new development efforts where they are generating up to 80 percent of the system’s code.
Today’s code generation engines are a better breed then those of years ago. They tend to offer more flexibility through the creation and use of simple templates that instruct the code generator on how to produce all the repetitive and highly structured code that developers would otherwise need to write.
Taking code generation to the next level, where code regeneration can also be productively used to maintain a system as it evolves, is a very challenging task that is normally only achievable if everyone buys into the use of code generation in the first place. However, such achievements are assisted by the current generation of code generators that use templates to provide more flexible output then their predecessors and .NET partial classes which store generated code for a class in separate files to that of its custom written code. Because the templates are parameter driven, small changes can be accommodated by rerunning templates though the code generator with a slightly different set of parameter values and replacing the custom generated code in the partial class files without touching any of the custom code for the class.
2.4 Being Forward Looking
The topics under this section discuss some items that any forward-looking system designer should take into account.
2.4.1 Scalability Versus Performance
The scalability of a system defines how responsive it remains overall while greater work loads (more users\transactions) are placed on it. A well-designed scalable enterprise system is capable of handling hundreds to thousands of people while still being capable of maintaining a good response to each individual.
Generally, enterprise systems are made scaleable by splitting them across tiers, which basically means running parts of the system on different computers dedicated to providing some specialized service or sets of data. While inserting additional tiers does tend to significantly and adversely affect the performance of any single request because of network latency and other issues, the ability to handle additional requests causes little additional system load because of the specialization in what they do. For example, if everyone logging on to your application requires a list of active stores, that information could be gathered from the database by an enterprise tier when the first person makes such a request, and it could then be cached on that tier, so it can be subsequently passed along to all others making the same request. The .NET framework itself also implements pools of shared re-usable resources such as database connections that users can be given access to as needed. When the user is done with the database connection it is returned to the shared pool where it remains ready to be used by the next person needing it. This removes the need for the system to incur the costly activity of creating a database connection every time someone needs one.
An application tier can be run on one or more computers in a load balance matrix (farm) like setup when necessary, to provide scalability to thousands of users. Such a setup can also provide fault tolerance for system, as the loss of any one instance of a tier does not stop the overall system from working.
The Modularization and Layering architectural patterns mentioned above are used to package tiers for system scalability.
2.4.2 Technology Usage
In building an enterprise level application, people should always be looking to the future, not only in regards to where the company might be going, but also as to the likely direction technology might take. Older technology offers less functionality, and is harder and more costly to maintain as time progresses.
Working with current technology makes its easier to find developers, free code, and productivity tools in a larger more active development community.
For example, if one was using the current technology of C# 2.0 and .NET 2.0 one would have access to freely available frameworks that would take care of much of the plumbing of any new system, a major cost and time saver. And the execution of smart business components that are easily re-used across Windows and Web Applications, and Window Services.
2.4.3 Long System Life
Systems always seem to live longer then most people expect. In fact, prior to focusing on being an enterprise developer, I completed a number of departmental level applications that were only expected to be run for six months, where just about all of them ended up lasting 3 to 5+ years. Had I actually programmed against the original estimate of their useful life, the client would have incurred additional cost to extend their life. However, the cost of programming for a much longer life was so small while writing the application, that I did it, which in the end turned out to be a major time savings, as opposed to me having to go back through another design, analysis, and testing cycle six months down the road, when I was no longer as familiar with the original requirements.
It’s even more important to plan for long life with a custom enterprise system built for a growing company, as the costs of retrofitting it later to extend its life is quite costly. Just as with small systems, its durability should be built with a long-term horizon in mind, not just a company’s present needs. It should be many, many years, before a business starts to stress the capacity of a newly developed system. It certainly shouldn’t be necessary to make apologies for a system’s shortcomings shortly after it’s been delivered, (or even as soon as it’s delivered).
2.5 Development Methodology
2.5.1 Iterative Development With Regular Reviews
Completing a detailed enterprise system design up front is an impossible task. Don’t let anyone tell you otherwise. Commonly, when interviewing business people, things are not always recorded correctly, or even interpreted properly. Not to mention, that people forget to tell you about important things because their not part of their everyday activity, or because they assume you know such information already. And also because they can’t easily visualize how a new improved process might be put together on a computer, (they are tool users not builders). In addition, as volumes of detailed information are gathered on a large system, designers tend to lose their ability to keep everything straight in their heads, and plan the perfect design. And its not uncommon for designs put together over a long period to be obsolete before they are implemented as the organization continues to adapt to changing business needs during the design phase.
The only really practical way to develop a large system efficiently is by gathering some high level requirements initially, and then moving on quickly to focus on individual functional areas\modules in an iterative development-cycle, soliciting regular feedback and guidance from actual business people so that regular adjustments can be made in a timely manner while the cost of doing them is still small. With regular feedback cycles and releases of working modules the final system is more likely to have what business users need.
2.5.2 Test Driven Programming and Continuous Integration
Test Driven Programming is a popular technique for ensuring the delivery of robust systems and maintaining them as such. It involves the construction of tests at the same time requirements are specified, where those tests are coded before the business functionality itself. As working code starts passing the tests, one gets assurance that the new functionality meets requirements. In addition, those tests can be executed in a testing harness with automated executions, which allow them to be run at anytime during the initial development process or any subsequent enhancement cycle to make sure that all versions of the system continue to pass all tests.
A complementary activity to Test Driven Programming is the concept of Continuous Integration, which set-ups automatic builds of systems from frequently checked in code, upon which tests are automatically run, and after which a final exception report is emailed to interested parties.
The loose coupling of applications and components described elsewhere in this document permits simpler tests to be created as fewer interdependencies need to be handled.
Some advocates of Test Drive Programming even view the tests as a means of providing some additional documentation as to what a system’s business rules are. However, I personally feel that such information would not be widely used and tests should instead likely make reference to particular rules in some better and more widely accessible classification of business rules.
Test Driven Programming and Continuous Integration can provide pragmatic tools for maintaining a robust system. Vendor tool support and company usage of these techniques is growing strongly.
2.6 Low Level Concepts
2.6.1 Flexible Deployment
The ability to deploy the system in a single tier (on a single PC) or in multiple tiers (individual user PCs plus one or more application, database, or web servers) should be easily configurable from an application’s file settings.
For example, if a remote sales office only has one user, then it might make best sense to run the module(s) he needs on a single PC. Other larger remote offices or facilities might need to run their modules across individual PCs, and at least one server for better performance. The main office with hundreds or thousands of employees may want to make use of separate servers for individual tiers to get good performance for everyone. All of this should be configurable from an easily modified application settings file.
It would also be highly desirable that the application itself be deployable over the web with a single action on the part of the user.
2.6.2 Provide Instrumentation Of Process And Data Pathways.
Using a single framework (or bus architecture) though-out a system provides the opportunity to provide instrumentation of individual processes and data pathways to identify who used what, when, how, and where. Such activity can help in trouble shooting performance problems and in identifying what parts of a system are no longer being used and can be retired
2.6.3 Components - Further Considerations
Some additional considerations that should be made when constructing components are described below.
Components – Promotion Of Abstraction In Design
Promoting abstraction while designing components can make them more flexible. If components are not unduly specific to any current set of steps or datum, they can be more easily enhanced as business needs change.
Components - Design For Behavior Not Data
Many developers new to Object Oriented Programming (OOP) tend to make the rookie mistake of creating too many simple classes that are largely only record set representations of all the tables in their database, a behavior that is encouraged by vendors of object-mapping-relation products, which will automate the creation of such objects. The biggest problem with having a data-centric view in performing OOP is that it tends to result in light weight objects that contain little intelligence. Which requires, that the smarts they should have contained be coded elsewhere in the system where its not easily reused, and therefore, it often ends up getting reproduced everywhere those objects are used.
Rather then simple data objects, it is better to focus on building business components that might represent data from a collection of tables and all the behavior that needs to be wrapped around them to perform some business task to promote code reuse. For example, an Order business component in its most simple form should be capable of holding customer summary data, such as their name, phone number and address, and a separate data line for each product they ordered, along with methods (behavior) to check that the customer has a good credit rating and calculate the time it will take to deliver their goods. If all this data and behavior is wrapped up in a single business component, then it is easier for Windows and Web programmers to build ordering tools that reference that single item to allow people place orders from both sources. And while the company needs to incur the cost of building two different types of interfaces they save money by only building one business component that does a good deal of the real business work.
Components - Normalize Behavior
Just as relational databases should focus on storing data in a normalized form (without repeating data groups) the practice of object oriented programming should focus on the normalization of behavior.
Taking our example of the Order component described in the section “Components - Design For Behavior Not Data.” it’s not necessary that the component actually know shipping times for products to a customer if the shipping system can already calculate that information. It would be better that such behavior only be implemented in one place, most likely the shipping system, and that the Order component would request a shipping schedule after passing the shipping system a list of products and their destination. The goal being, that actual behavior gets coded in only one place, and therefore, there is only one place it ever needs to be updated when changes are needed. And while the actual calculation of a shipping date\schedule could be done elsewhere, users of the Order component do not need to be aware of that. They would just ask the Order component directly for the estimated ship date, whether the component calculates that information itself, or passes the request along elsewhere, should not be of concern to anyone using that component.
2.6.4 Simplify Format Of Data Storage And Manipulation
It’s desirable that data be passed around the system in a fairly rudimentary format that is largely independent of any vendor specific data management technology that is likely to change every few years, as it has done for Microsoft and Borland over the last 15 years. This helps reduce an application’s dependence on any software vendor’s more specialized data management technology, making it easier to replace in the future.
For example, large amounts of table like data in .NET components can be kept in simple tabular structures with rows and columns that support the .NET Binding List interface that provides data binding for forms permitting bidirectional data exchanges between both. This can be done in preference to the much more sophisticated data typed xml datasets, which are much more likely to be subject to change in each new release of .NET and eventual replacement within two to three years when the vendor creates a new data access technology. The simpler data structures may change too, but generally those changes will be small and manageable, and possibly not affect the business components at all. Even better, if the behavior around those simple structures is coded in a common ancestor for all business components, then any adjustments that need to be made only need to be adjusted for in one place for all components, which would not be the case for more complex data structures, as they get implemented as independent entities with no common ancestor. Another advantage of the simpler structure is that it’s easier and faster to pass data from one component to another for any additional processing, as the data’s storage container carries much less baggage with it.
3 Conclusion
Building enterprise systems is an inherently complex business. Getting all the requirements and design right at the beginning of a large system never happens. If what was presented in this document didn’t persuade you of that, consider the fact, that no one has the rights to a working crystal ball to predict the future, and stuff happens. So by necessity, system development has to be done while trying to hit a moving target. Change is the only constant in our lives, small changes happen all the time, more fundamental changes occur as time progresses.
In accepting the above facts, this document has shown that one can be proactive about planning to handle change when building large systems. It explained how the development effort will be much easier and ultimately more successful if one selects methodologies, tools, and techniques to craft an adaptable development process that is responsive to changes. Having an adaptive development process, allows the development personnel to make necessary changes while development is underway, while it’s still cheaper to do so, and to produce a system that is better geared to what the business’s needs are at the time of delivery.
Some of the most important guidelines discussed in this document for providing an adaptable system were: 1) Making use of blueprints, domain models, and standards. 2) Adopting proven software patterns. 3) Using coding techniques such as component building, interfaces, meta-data, and code generation. 3) Being forward looking with system scalability, performance tradeoffs, and technology usage 4) Using the Agile methods of Interactive Development, Test Driven Programming, and Continuous Integration. 5) Providing for flexible deployments, instrumentation to monitor system usage, and being careful to adopt a fairly neutral method for data storage and handling of data.
While this document did cover a number of major areas of knowledge that any enterprise software architect\developer should be familiar with, it’s not intended to be a comprehensive list. Overall, software architecture is a pretty large and complex subject area offering many challenges. For those interested in the subject, the References section below provides additional sources of information on it.
In larger companies, the software architectural load can be split amongst architects by allowing them develop specialties from which they can contribute unique knowledge to the overall team effort, while only being required to have a general knowledge about other areas. In smaller companies, the software architect generally needs to cover broader ground in less depth, as the company isn’t often large enough to have someone dedicated to work on that activity full time. In such cases, the architect often also needs to manage the development staff, do project management, and actually help write the software. To compensate for being spread so thin, certain responsibilities need to be pushed further down the line to other development personnel who are willing to accept and learn new skills to help make the enterprise development effort a reality. Skills such as those outlined in this document.
4 References
Analysis Patterns – Reusable Object Models by Martin Fowler, ISBN 0201895420. Provides a catalog of “business domain” patterns.
Applying UML and Patterns—An Introduction to Object-Oriented Analysis and Design by Larman, ISBN 013748807. Provides a foundational discussion for building robust, scalable, maintainable systems, using object technology.
Building Business Intelligence Applications with .NET by Robert Ericsson, ISBN 1584502711. A practical guide to using .NET tools to build applications for OLAP and data mining.
Code Generation in Microsoft .NET by Kathleen Dollard, ISBN 1590591372. Teaches code generation as a scriptable and repeatable process using templates so one is not tied to a particular framework or style. Kathleen is one of the industry leaders and a major advocate of code generation. Her book also has a couple of chapters on code generation for the CSLA framework (refer to the “Expert C# Business Objects” book below).
Component Software – Beyond Object-Oriented Programming by Clemens Szyperski, ISBN 020117885. It’s about developing re-usable off-the-shelf components that handle a significant amount of functionality. It emphasizes that one should not focus on trying to build each new piece of functionality from collections of small independent objects while developing larger systems. One should instead, focus on developing more robust components that wrap up smaller objects into bigger containers that are capable of performing bigger units of work, which in turn, require less effort by other developers to reuse them to build even bigger functional units.
Design Patterns – Elements of Reusable Object-Oriented Software by Erich Gamma, Richard helm, Ralph Johnson, and John Vlissides, ISBN 0210633612. This book is the original classic on software design patterns, and still the best reference on the subject.
Domain Driven Design by Eric Evans ISBN 0321125215. This book makes a persuasive argument for the use of domain models to provide a ubiquitous language that ties the domain experts and technologists together. Also, it emphasizes that really powerful domain models only evolve overtime and that some of the best ideas appear after the initial release of a system.
Enterprise Integration—An Architecture of Enterprise Application Systems Integration by Fred A Cummins, ISBN 0471400106. An excellent book, that reviews the technology landscape defining enterprise integration objectives, and which provides generalized enterprise integration architecture.
Enterprise Integration Patterns—Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe and Bobby Woolf, ISBN 0321200683. This book describes using asynchronous message as a proven strategy for enterprise integration, and offers 65 patterns for building such systems.
Expert C# Business Objects by Rockford Lhotka, ISBN 1590593448. A book about application architecture, design, development, and using object-oriented concepts in .NET. Book describes the building of a “Component-Based, Scaleable, Logical Architecture” (CSLA) in C# .NET, which has been used as the basis of system development efforts in many companies with good success. Also, the company Rocky Lhotka works for relies on his framework and code generation to produce upwards of 80% of the code in new systems they develop for clients.
Object Technology – A Manager’s Guide, 2nd Edition by David A. Taylor, PH.D., ISBN 0210309947. This book is an easily read classic on the subject. One of its central themes is that objects are not about the data they contain, but the abstracted behaviors they represent.
Patterns of Enterprise Application Architecture by Martin Fowler, ISBN 0321127420. A book written by a well known industry expert on the topic of suitable patterns for enterprise architecture. The book is written in two parts, the first part is a short tutorial on developing enterprise applications. The second section and bulk of the book is a detailed reference to patterns that assist in building enterprise applications.
Pattern Oriented Software Architecture—A System of Patterns by Frank Buschmann, Regine Meunier, Hans Sommerland, and Michael Stal. An excellent book concentrating on Architectural Software Patterns such as: Layers, Pipes and Filters, Distributed System Broker, Model-View-Controller, and Microkernel.
Strategic Data-Planning Methodologies by James Martin, ISBN 0138511136. The first good book I ever read on the topic of strategic data planning and still one of my favorites.
