Summary
Developing Web applications is difficult due to conflicting demands for high quality applications built in Web time, and to immature, unintegrated, and constantly changing technologies. Web application design patterns help manage this complexity and indicate the required tools. Tools need to be integrated and manage the relationships between their associated artifacts. A tool integration platform provides the required "integration-ware" enabling the development of products consisting of integrated, configurable, roles appropriate, and end-to-end development tools based on best practice design patterns. To reduce acceptance barriers, this integration-ware must be based on open standards as much as possible and Open Source where standards don't exist. To be effective, tools need to be integrated into a common workbench. The workbench and integration-ware must support flexible tool integration appropriate to vendor business objectives by providing various conformance levels of integration. Conformance levels define what any vendor must do to integrate at a level, what technologies are required, and any integration expectations.

The Web is changing everything

Customers are discovering the opportunities and challenges of e-business applications. Current integrated suites of office tools like Microsoft Office and Lotus SmartSuite set high expectations for integrated application development tools. The Internet is changing the face of business integration resulting in rapidly changing business models due to BPR in order to leverage the Web. These e-business applications can be complex, difficult to maintain, and demand interoperability. Interoperable applications consisting of highly integrated artifacts demand integrated tools supporting a prescriptive methodology in order for application development to be a rational, controllable, and repeatable process.

Web Application development results in the need for layered tools

The explosion of Web technologies including client display architectures, communication protocols, application servers, and backend systems has resulted in rapidly changing and complex application architectures. The Web represents a complex, highly dynamic, rapidly changing application development problem space. A Web application incorporates many resource types representing diverse, highly interrelated components including: HTML, GIF, JPEG, DHTML, scripting, Java Applets, ActiveX controls, servlets, EJBs, connectors to legacy systems, etc. In most cases, these resources are never collectively compiled in order to validate the intended relationships. This can result in applications that are fragile, difficult to test, scale, and maintain.

Because of these diverse content types, and their contents, building a Web application requires diverse, highly specialized development skills including: GUI programmers, server programmers, graphic designers, database designers, business experts, and document designers. Client web browsers and HTTP web servers establish the overall architecture of Web applications. The resulting distributed applications require compromises between accessibility, availability, integrity, ease of use, performance, and footprint. Web applications are targeted at a broad audience, which requires emphasis on ease of use, appealing look and feel, and masking resource and distribution technologies. Web application development is pushed from two conflicting directions: the need so support high quality applications, and the need to develop them very fast on emerging and un-integrated technologies.

The techniques for developing Web applications are changing fast. This often results in new and immature technologies and many tools: new tools, new revisions of old tools, and beta releases of emerging tools. As a result, there is a significant opportunity for point tools supporting these new emerging technologies that facilitate the development of Web applications. However, these tools must be effectively integrated in order to speed Web application development, provide end-to-end debugging, minimize gaps and overlaps between tools, and provide a positive developer experience.

Patterns help manage the complexity

Fortunately, web application development choices tend to fit into patterns based on application functionality and network, middleware, and runtime topologies. Patterns help focus on customer needs, the technologies, tools, and techniques that are used to build web applications. Each pattern is instantiated through the creation or reuse of a number of web resources. Each resource requires a tool to author the resource and a repository for storage. Resources are integrated and linked together to support relationships existing in the target business application. Tools must be integrated in order to facilitate development of an application built from integrated artifacts. Otherwise the developer must maintain these relationships manually. This is both tedious and error prone.

Effective tool integration requires open standards, a component and programming model, frameworks, APIs, file formats, etc., collectively called "integration-ware". Integration-ware provides the base platform in which extensions can be applied to provide domain specific tooling. All these components must be integrated with a central workbench that provides access and control of the entire site across all resource types and application development roles. The benefits are reduced gaps and overlaps between tools, elimination of repetitive, manual, tedious, and expensive on-site tool integration, less redundant data input, and a more positive user experience.

Realizing the Tools Opportunity

Web and e-business application development, and a tool integration platform result in a significant opportunity for ISVs to develop application development tools. In order to be successful, tool vendors need to focus on the value added by their tools while minimizing their investment in infrastructure, porting to other operating systems, browsers, and/or web servers. ISVs also need to minimize the amount of ancillary functionality they must be develop in order for their tools to be accessible and useful. Minimizing this investment involves integrating tools with other complementary tools and IDEs in such a way that the integrated result provides more value to the end user than the individual tools by themselves. However, integrating tools can require a significant investment. Each tool or IDE that is a candidate for integration provides its own set of extensibility and integration mechanisms. In order to achieve maximum market penetration, a tool vendor will want to integrate with as many other tools and IDEs as possible, and in such a way that the tools compliment each other without creating undue redundancies and coupling.

Eclipse provides the required tool integration framework

A tool integration platform is required to exploit this tooling opportunity by providing a platform for deploying integrated, configurable, roles appropriate tools for developing end-to-end, Web-based, e-business applications. As well as enabling tool integration, a platform helps minimize the cost of tool integration by providing a large number of reusable libraries, APIs, and frameworks that enable provide common services. This minimizes the tool vendors development costs and allows them to focus on their value add.
Wherever possible, the platform must use open standards like WebDAV, XMI, JavaBeans, etc. to remove barriers to acceptance, limit tool vendor investment, and reduce time to market. In those cases where there either is no standard, or there are multiple conflicting standards, Open Source can be exploited to establish momentum within the development and tool communities to define defacto standards.

A Workbench Provides Flexibility and Centralized Project Control

The tools required to build an application must vary in order to match the customer's preferred application patterns. There are three ways developers can use tools to develop applications:

1. Simple applications can be built using independent, specialized tools with all component relationships and publication managed by the developer.
   
2. An IDE can be used to provide all the integrated tools required to build the entire application.
   
3. Use a workbench to integrate tools matching customer needs.
   

The first approach has the advantage that it provides hands on development where the developer knows everything. Developers can choose any tool they want to build any resources they want. However, this approach doesn't scale to larger, more complex, highly linked projects requiring multiple versions, and a larger development team. It becomes increasingly difficult for a single individual to comprehend the complete project. Taking this approach can often result in applications that are hard to understand, extend, validate, and maintain.

The second approach is appealing because all the tools required for building and maintaining applications are incorporated into a single product. The developer gets everything needed in a single product from a single vendor. This presumably results in well-integrated tools. However, these IDEs are extremely difficult to build because there are so many component types, and technologies are changing too fast. IDEs can be sufficiently complex that they cannot keep up with changes in application domains, deployment architectures, technologies, or development processes. It is difficult for an IDE to match the features of a best-of-breed point tool resulting in mediocre tools and poor user experience. Users do not have the flexibility to choose and integrate the tools in a manner that best matches their preferred development style and e-business application pattern.

The workbench approach solves these problems by providing a central integration point for project control and an integration mechanism to include resource-specific tools. This approach allows a user to build applications using any components they want with any tools while at the same time providing a common view of the complete application across all components and the entire team. This allows flexibility and choice necessary to select and integrate tools that meet different e-business patterns and developer roles but still keeps the application under control. The workbench can provide services common to, or across tools including reporting capabilities, managing relationships or dependencies between components, component management, and publishing services. Using a workbench simplifies tool integration by allowing tools to integrate with the central workbench instead of each other. This may significantly reduce the number of integrations that must be developed, and provides a more consistent environment for the end user. The workbench can be configured with the tools that best map to the e-business development patterns.

The combination of a workbench and integration-ware for developing integrated tools is a win-win situation for platform vendors, ISVs, and application developers. Platform vendor(s) provide integration-ware and services enabling the efficient development of integrated tools. ISVs contribute to the portfolio of integrated tools while minimizing their costs through reuse of the platform and complimentary components produced by other ISVs. Business application developers have best-of-breed tools integrated together supporting their development process.

Integration Conformance Levels

In order to provide flexible tool integration, a tool integration platform must define a number of conformance levels that tool vendors can target based on the desired level of investment, time to market, and specific tool needs. Each level specifies what tool vendors must do to achieve that level of integration, and the required technologies that must be employed. By defining these integration conformance levels, tool vendors have choices while at the same time specific rules for advertising conformance to a particular level. This makes it easier for tool vendors to determine what they must do to integrate their tools with the platform, what technologies have to be supported, and establishes a consistent, deterministic end-user experience. The integration conformance levels are:

None: No integration, tools are separate and independent. This is fine for situations where tools operate on independent resources that are not linked or otherwise integrated. It is typical of tool integration today, and often, no further integration is needed.

Invocation: Integration is through invocation of registered applications on resource types. Most operating systems and IDE provide this level of integration. A user or installation program can register an application on a particular resource type, and when that resource is selected, the application is invoked on the selected resource in a separate window. There is no further integration. Invocation integration is better than none because it allows the user to browse a collection of artifacts in a file browser, Web browser, or other tool and invoke the correct application to author the resource.

Data: Integration is achieved through data sharing between tools. This is the type of integration that was attempted in the past with technologies like IRDS, PCTE+, and the IBM ADCycle program. In this level of integration, tools share each other's data. This provides much better integration than simple tool invocation because now the tools know about each other's resources and can manage relationships between them. Data integration consists of there basic concepts: access, interchange, and transformation. Access is the protocol or API used by the tool to physically access the data. Interchange is the form the data takes once it is accessed, i.e., its schema. Transformation is concerned with modifying the data to meet the needs of the target tool when the interchange format or schema isn't in the form the tool requires. Transformation can also be used to integrate date from many tools into a new single data model meeting the needs of some new tool.

Data integration between tools creates a combinatoric explosion of access, interchange, and transformation protocols. In the past, we tried to solve this problem by specifying a repository and schema for all tools to integrate with. This turned out to be impossible because tool vendors couldn't let a single vendor (often a competitor) control such strategic aspects of their infrastructure, and no one repository or extensible schema was sufficient to meet all vendor needs. A better solution is exploit open standards that isolate tools from each other and their underlying repositories and schemas. Fortunately there are a number of standards that exactly meet our needs. WebDAV (Web Distributed Authoring and Versioning)

using WebDAV for data access, XML/XMI/MOF for data interchange, and XSLT for data transformation and integration.
API: Tools interact with other tools through JavaBeans that abstract tool behavior and make it publicly available.
UI: Tools and their user interfaces are dynamically integrated at runtime with a common desktop.
The Workbench Tool Integration Platform is Different
Then we need to establish a set of patterns that describe typical architectures used by ISV tools. These might include:

Java/Swing application using files in the file system for persistence
OLE Document editor
Java/Swing application using a non-file based repository (object repository or relational, doesn't really matter which)
C++ client/server application with proprietary client/server communication protocol and proprietary repository
etc.

Then we look at each one of these patterns and explore how they can transition to Workbench technologies through the levels of integration. What we'll see is that there are patterns here too. Things like:
Launching the application client on a resource type
File based tools can use the workbench resource management model versioning, access control, synchronization, disconnected work, etc.
OLE Document editors can easily integrate their UIs, but need to do additonal adaptor work to integrated with the outline, properties, and task views.
Java/Swing applications can run in the same JVM so they can access other plug-in API (other tools' busines models).
Java/Swing applications can be easily converted to Desktop editors by modifying their menu contribution to use either Swing menus or desktop contribution depending on who launches them. They can also put their top-level component in an SWT container so they can easily integrate with the rest of the workbench, similar to ActiveX controls.
Tools with proprietary repositories can start by exporting data out of their repositories as XMI document through queries that represent typical units of work. These XMI documents can be consumed by new tools integrated with the workbench while existing C++ based clients work on the same repository.
The next step for these tools would be to provide an interfact to their tools model in a Java plug-in. The Java model could use the XMI documents, the repository directly, or both for persistence. Using the new editor extension-points, this Java model could be used to develop new editors on the tool's model while maintaining the same repository.
The repository could implement a WebDAV interface to provide a standard protocol for accessing the XMI documents.
and so on.