> Technology White Papers > Standards-Based Technologies 8.5

Standards-Based Technologies and Enterprise Messaging Infrastructures

© 1997 Ron Herardian, Global System Services Corporation (GSS), and Mike Faul, Netscape Communication Corporation

OVERVIEW

Corporate CIOs and IT architects are taking a hard look at open Internet standards-based electronic-mail systems as the intranet alternative to traditional proprietary email systems. Most corporate email systems comprise one or more proprietary systems that coexist through the use of email gateways but do not comply with the same standards or features, or interoperate transparently. Coexistence of multiple proprietary email systems is technically problematic and costly. For many companies, this mix of email systems becomes a money pit that consumes hardware resources, network bandwidth, and staff.The current move to corporate intranets signals a categorical rejection of proprietary infrastructures for business information systems. Although the industry press tends to focus on electronic commerce, so-called "groupware," and the World Wide Web rather than on messaging, email is an important factor in the move toward intranets. Today, companies are looking on proprietary email and groupware products in the same way firms implementing local-area networks (LANs) regarded mainframes and minicomputers during the late 1980s. Of course, with mainframes and minicomputers, there was a difference in scale, if not in end user functionality, compared with microcomputer LANs. In contrast, standards- based email technologies have deep technological roots and represent the most scalable email technology.

PROPRIETARY AND STANDARDS-BASED EMAIL TECHNOLOGIES

Email is as essential for businesses communications as is the telephone. Unlike most telephones and the public telephone network, however, email programs and servers from different vendors commonly don't work together. Since the Internet has become email's public telephone network, email client applications and servers should be compatible with Internet standards. Unfortunately, for historical reasons, most corporate email systems are not based on Internet standards. However, in recent years, Internet standards have increasingly defined corporate email technology, and technologies are now available that enable companies to move beyond costly, proprietary systems.While proprietary systems are built on technologies owned and controlled by a single vendor, standards-based systems are built on technologies owned by no one and controlled by a standards development process open to all vendors. Although proprietary email systems have been successful in the past, most corporate CIOs and IT architects are now strategically committed to open standards-based technologies. Standards-based email systems facilitate interoperability among vendors, giving customers a wide and flexible choice of email client applications and servers.In the past, proprietary systems offered corporate customers security, more features and functionality, and the ability to rapidly develop new features; however, these advantages came at a price. LAN-based proprietary email systems, for example, were not scalable: Either they couldn't accommodate large numbers of users or the cost of monitoring and maintaining larger systems was prohibitively high. While in the past proprietary meant better security and more features, it also meant basic incompatibility with other proprietary systems. Incompatibility, in turn, meant a loss of security and message fidelity between systems, as well a proliferation of email gateways and their corresponding system-monitoring and maintenance challenges.Standards-based email technologies have been slower to evolve than proprietary systems, but they have deeper technological roots. Today, standards-based email systems deliver the advantages of proprietary systems without their drawbacks. As a result, vendors of proprietary email systems have rushed to extend their products to support open standards in addition to their own proprietary technologies. Although vendors of proprietary email systems originally resisted supporting Internet standards, they are now being forced to play catch-up having lost business to vendors offering standards- based products.While standards-based products make up the Internet and corporate Intranets, vendors of proprietary email systems have effectively 'bolted on' a series of add-ons that allow their products to incrementally approach the goal of interfacing with Internet standards - an inevitable approach but one with intrinsic drawbacks. The most obvious problem for vendors of proprietary email systems is the accelerating trend towards ever fatter, slower clients and resource-hungry servers. At the same time, an influx of new vendors offering standards-based products has accelerated the Internet standards development process, and these vendors are introducing new features and functionality at an unprecedented pace. This situation leaves vendors of proprietary systems one step behind in technology while their customers must pay more for email due to bloated infrastructure costs.

PROBLEMS OF PROPRIETARY EMAIL SYSTEMS

The high costs and other problems of proprietary email systems can be attributed to a number of factors, including email gateways and message switches, reliability concerns about gateways and monitoring mechanisms, enterprise wide directory services, security and message fidelity, cross-platform feature parity, and incompatible application programming interfaces (APIs) used by mail-enabled applications at the desktop - all of which can be solved by open Internet standards-based technologies and products.

Gateways
Security, fidelity, and reliability across platforms and email systems can be ensured in one of two ways: by maintaining a variety of email gateways and the staff to monitor and maintain them, or by depending on a single messaging infrastructure. However, if a company bases its enterprise email infrastructure on a proprietary email system, none of the issues outlined above can be resolved when it comes to enterprise and Internet communications. In other words, a company can solve many of the problems associated with proprietary email systems in house by simply standardizing on a single-vendor solution, but those solutions will not extend to communications beyond the enterprise.

Message Switches
Some vendors of proprietary email systems claim that because their servers support multiple proprietary email systems, their gateway modules are really native message transfer agents (MTAs). However, any software that translates and transmits from one kind of email system to another is by definition a gateway. The trend toward multiple MTAs in proprietary email servers signifies that these vendors have introduced low-end message switches in an attempt to tackle the problems of multiple proprietary email systems. As this white paper will show, the correct technological solution is not to build servers that support multiple proprietary email systems but to instead introduce standards that provide equal or superior levels of functionality.

Monitoring
Monitoring multiple email systems and gateways is technically challenging because proprietary email systems and their associated gateways do not support common monitoring mechanisms such as Simple Network Management Protocol (SNMP). In fact, most proprietary email systems have very limited monitoring capabilities - especially LAN-based email where email "post offices" are deployed over multiple file servers. The situation worsens when multiple proprietary email systems are in place because of the added challenge of monitoring disparate email systems and a number of third-party gateways.

Reliability

Although some proprietary email systems reliably transport messages within the system, the mechanisms employed to achieve this do not operate across systems. Email gateways are often vendor afterthoughts or are written by third parties, and reliance on them makes multi vendor email systems unnecessarily complex and correspondingly fragile. In addition to gateway issues, LAN-based email systems may depend on workstation-based MTAs and other network components that are not contained within the email server.Directory Services
In addition to ensuring the security, fidelity, and reliability of messages throughout the enterprise, companies need to maintain common directories. However, larger companies often have disparate and incompatible email systems that rely on email gateways and proprietary directory- synchronization products to tie them loosely together. Proprietary email systems use proprietary directories. Thus, customers must either settle for limited integration with LAN directories (such as NetWare or Windows NT directory services) or buy a proprietary message switch that provides a directory-synchronization facility across email systems. Neither solution is adequate, and both require significant investments in proprietary technologies.

Security
Within a single proprietary email system, security is normally good because messages can remain encrypted - both during storage and in transit from one point in the system to another. However, this does not mean that there is no risk of unauthorized access by trusted administrative staff or that there are no systemic security exposures that do not fall within the scope of message storage and transfer. Even more important, cross-system security has been virtually nonexistent when transferring messages across multiple proprietary email systems or from a proprietary system to the Internet. In other words, when messages are sent outside the proprietary system, they are either vulnerable to unauthorized access at some point or transferred in a non secure way.

Fidelity
With multiple proprietary email systems, either within an enterprise or when exchanging email with outside organizations, messages are almost never delivered with their formats intact. This can mean loss of data, such as embedded graphics or file attachments, or of text attributes like boldface and italic type. With multiple proprietary email systems or when sending messages from a proprietary system over the Internet, users prepare messages only to have their formats altered or lost by a gateway or rendered unreadable by recipients. Vendors of proprietary email systems have promised customers Rich Text Format (RTF) as a potential multi vendor standard; however, RTF has not garnered wide support nor do email gateways support it, nullifying its value as a multi vendor standard. Messages delivered from one proprietary system to another often lose more than message text attributes and format, file attachments, and embedded graphics - not surprising when you consider that messages must pass through a series gateways from different vendors that don't necessarily comply with the same standards. Typical problems include loss of address information and audit trails, loss of delivery and return receipts, and loss of message priority attributes. No proprietary vendor has addressed the need for message text and attachment standards.

Feature Parity
Vendors of proprietary email systems typically support two or more platforms but most proprietary products have a heavy Windows bias. Typically this means that versions of email products on non-Windows platforms suffer a loss of features, slower product releases, quality problems, and poor performance due to porting of Windows-based code to other platforms. The platform disparity on the part of proprietary email systems reflects a lack of focus on the platform-independent standards and technologies that would be of substantial benefit to customers.

Proprietary APIs
Some vendors' cross-platform strategies depend in part on APIs - such as Vendor Independent Messaging (VIM) and the Mail API (MAPI) - that are not necessarily supported (or equally supported) on all platforms. Proprietary APIs prevent customers from choosing the best products in each category because they are restricted to the use of products that support whatever APIs are available for their proprietary email system. This also means that add-on products (such as business forms and calendars) that work on one email system within the enterprise may not be able to interoperate with equivalent products running on other email systems within the same enterprise. As a result, each proprietary email system represents an island of isolated add-on functionality made up of virtually arbitrary products supporting the available APIs.

All vendors of proprietary email systems want corporate customers to believe their infrastructure is the right one. The result is corporate email systems that often represent a mishmash of incompatible systems strung together with a series of hard-to-manage gateways. Underlying the current situation are two basic problems: the historical failure of proprietary systems to scale up to large enterprises and, even more importantly, a simple lack of standards.

Open Internet standards-based technologies address all of the issues outlined above - without forcing companies to standardize on a single, proprietary infrastructure. This in turn provides corporate email users with flexibility in terms of servers and clients while eliminating the problems of proprietary email systems. A standards-based email infrastructure provides improved manageability and lower cost of ownership.

GUIDE TO STANDARDS-BASED EMAIL SOLUTIONS

Internet standards are ratified by groups of individuals from interested companies, and the standards process is open to all concerned vendors. The working groups - which make up standards bodies such as the Internet Engineering Task Force (IETF) and the Worldwide Web Consortium (W3C) - develop standards and methods that can be implemented consistently. While competing vendors of proprietary email systems have in the past been unresponsive to customer concerns regarding interoperability, several key Internet standards have emerged that address the issues outlined above. All of the technologies described below are currently available and based on open standards that work transparently across vendors and computing platforms.

User Message Delivery POP3 and IMAP4
Rich Text HTML
Embedded Links HTTP
Message Transfer and Security SMTP and E/SMTP
Encrypted Mail S/MIME
Certificates X.509
Directory Services LDAP
System Monitoring SNMP
 

 

USER MESSAGE DELIVERY

As the forerunner of IMAP, POP3, or Post Office Protocol, allows client applications to authenticate, list, and download messages stored on email servers. Serving as a rich protocol for accessing remote message stores, IMAP4 provides mechanisms for accessing public mailing list archives and private and shared message stores as well as synchronization for mobile and disconnected users. Synchronization is defined as an email client's ability to maintain the same folders and contents on multiple machines (such as office and home computers) regardless of hardware or software. IMAP's selective download capability allows users to connect to an email server and acquire a list of new messages, which the user can then select and download as desired (including or excluding file attachments). IMAP's selective download capability reduces the time that users are connected to the server and, by minimizing traffic, increases the overall performance of the server and network. Efficient server resource utilization also means greater numbers of users per server. HTML is the standard document format and method for rendering within a Web browser. In the context of email, HTML controls such message format characteristics as color; character size; type characteristics such as bold, underline, and italics; fonts and type styles; positioning; and the embedding of nontext objects such as graphics, animation, and embedded Java applications. Users have long wanted to be able to create and deliver messages that can be viewed as originally intended, regardless of system. However, in the past, it was impossible to make the transaction seamless to the user while still maintaining the content's format. HTML solves this problem.

HTTP is the protocol by which HTML pages are transported from a server to a client on the World Wide Web or corporate intranet. HTTP can also be used within an email client to obtain information from web sites or documents linked within an email message.

MESSAGE TRANSFER AND SECURITY

SMTP and E/SMTP are TCP/IP-based protocols used to transfer messages between email servers within an intranet or over the Internet. These small protocols were designed to allow the rapid transmission of message data across wide-area networks or internal enterprise networks. As the most scalable core email technology in the world, SMTP can handle the entire Internet network.

S/MIME provides an end-to-end public-key encryption mechanism in which a message encrypted by the sender can only be decrypted by the recipient. At no time during the transmission or routing of the message is the message stored unencrypted or can any user or administrator access its content. S/MIME provides private messaging with sender authentication and tamper detection. When a user creates a message, its content is encrypted with the user's public key. The message is then transmitted (encrypted) to the recipient whose email program decrypts the message with his or her private key.

X.509 certificates act as electronic passports, granting access to the email server or applications running on various systems. To log in to a system, a user's client application presents his or her certificate to the system, which then uses it for authentication, access control, and digital signatures. Information designed for external users such as business partners would be available only to users possessing a certificate issued by the organization for that purpose. Not only can system administrators revoke certificates manually or automatically, but in some cases mechanisms exist that allow organizations to retrieve information that may have been encrypted maliciously via key escrow (a mechanism that allows key issuers to maintain a copy of the stored key).

As the standard method of handling file attachments, MIME defines a data type for each data file format or application so that attachments can be accessed transparently across platforms - even if the recipient is using different applications. MIME includes character-set information so that non- ASCII characters can be transferred within the message text without any loss of message fidelity.

DIRECTORY SERVICES

A large part of any scalable messaging infrastructure is its ability to address a message to any user in the system. Users should be able to access an electronic directory and send messages without having to know what messaging system the recipient is using. Directory information must be communicated across a network from any email system to any user regardless of the email clients and servers used. Directory information should also be synchronized throughout the enterprise so that all users have access to the same information regardless of what server they're accessing. At the same time, changes to directory information made at the departmental or division levels of an organization should be propagated throughout the system.

LDAP, or Lightweight Directory Access Protocol, is a subset of the X.500 directory system's DAP (Directory Access Protocol). In the past, some companies have stored system-wide directory information in X.500 directory systems. These systems conformed to the X.500 standard, but their complexity and size meant that the capabilities of X.500 were not available directly to users. Designed to allow applications access to large distributed directories, LDAP allows organizations to maintain directory information in any system that supports the LDAP standard. Through cross-vendor LDAP support, directory information can be shared across systems, eliminating the need for proprietary message switches or directory synchronization gateways. From an administrative viewpoint, LDAP provides a single point of administration or distributed ability to modify directory information. LDAP can integrate diverse directory services and protect existing investments in directory technologies such as X.500, NDS, Windows NT, and so on.

DISCUSSIONS

NNTP, or Network News Transfer Protocol, defines a core groupware technology where threaded user discussions are replicated between servers. Email and discussions together form the backbone of groupware technology. In the messaging context, NNTP can be used to create newsgroups similar to the bulletin boards and discussion databases of proprietary email systems. In proprietary email systems, NNTP is integrated through a gateway that synchronizes newsgroups with that system's shared message architecture. In contrast, standards-based products can build in native NNTP support, eliminating the need for gateways or separate news reader applications.

MODEL FOR STANDARDS-BASED EMAIL

The figure below represents a simple standards-based messaging system that supports remote and local mail clients accessing a common message store using either the POP3 or IMAP4 protocol and LDAP directory services. Directory information is retrieved by the user's email client application from the LDAP server. In the scenario shown, a user can create messages, find the recipient's address, and send the message to the server. Virtually all standards-based email client applications allow users to store addresses in a personal address book for later use.

 

 

With IMAP4, a remote client can connect to the mail server, which lists waiting messages. After downloading the message list, a user can decide which messages or parts of the message to retrieve. With POP3, on the other hand, users would download all pending messages. Using IMAP4, email users can organize messages into folders that are synchronized automatically with the server. When a user connects to the email server from any client, his or her folder hierarchy can be rebuilt.

SUMMARY
Coexistence of multiple proprietary email systems and gateways is no longer necessary now that standards-based products offer the advantages of proprietary systems without the drawbacks. As corporate IT infrastructures move toward Intranet technologies, the proprietary email technologies of the past can be systematically replaced by standards-based systems. Open Internet standards-based messaging technologies deliver highly scalable, feature-rich, secure email to the corporate enterprise. Standards based messaging technologies solve the major problems associated with proprietary email systems while providing maximum flexibility, compatibility, and scalability to customers.

ABOUT THE AUTHORS
Leading email and Internet expert Ron Herardian of Global System Service Corporation (GSS) has written numerous technical papers and articles on electronic messaging and he has been a popular speaker at industry events. GSS clients include numerous Fortune 500 companies and U.S. government agencies. Herardian is currently the CEO of GSS, a Silicon Valley-based consulting firm with offices in the Silicon Valley California providing networking, electronic messaging, groupware, Internet and intranet, and World Wide Web consulting services. GSS is a Qualified Lotus Business Partner, a Microsoft Certified Solution Provider (MCSP), a Principal Partner in the Sun Partner Advantage program and a member of the Sun Software Partner Council a member of various industry organizations.

Mike Faul of Netscape Communications has been involved with messaging and directory issues since 1985 and has written several articles related to the coexistence and scalability of messaging and directory systems. He has more than 10 years of experience and has been a speaker for several industry trade shows. As a principal systems engineer with Netscape, he is one of the company's key advocates for large-scale messaging and directory issues and has designed Netscape's scalable messaging architecture.

 

APPENDIX: MESSAGING-RELATED RFCS

RFC2046 MIME Part Two: Media Types
RFC2045 MIME Part One: Format of Internet Message Bodies
RFC2044 UTF-8, a transformation format of Unicode and ISO 10646
RFC2034 SMTP Service Extension for Returning Enhanced Error Codes
RFC2033 Local Mail Transfer Protocol
RFC2017 Definition of the URL MIME External-Body Access-Type
RFC2015 MIME Security with Pretty Good Privacy (PGP)
RFC1985 SMTP Service Extension for Remote Message Queue Starting
RFC1947 Greek Character Encoding for Electronic Mail Messages
RFC1927 Suggested Additional MIME Types for Associating Documents
RFC1895 The Application/CALS-1840 Content-type
RFC1894 An Extensible Message Format Delivery Status Notification
RFC1893 Enhanced Mail System Status Code
RFC1892 The Multipart/Report Content Type for the Reporting of Mail System
RFC1891 SMTP Services Extension for Delivery Status Notification
RFC1874 SGML Media Types
RFC1873 Message/External-Body Content-ID Access Type
RFC1872 The MIME Multipart/Related Content-type
RFC1870 SMTP Service Extension for Message Size Declaration
RFC1869 SMTP Service Extensions
RFC1864 The Content-MD5 Header Field
RFC1854 SMTP Service Extension for Command Pipelining
RFC1846 SMTP 521 reply code
RFC1845 SMTP Service Extension for Checkpoint/Restart
RFC1844 Multimedia E-mail (MIME) User Agent checklist
RFC1820 Multimedia E-mail (MIME) User Agent Checklist
RFC1815 Character Sets ISO-10646 and ISO-10646-J-1
RFC1767 MIME Encapsulation of EDI Objects
RFC1741 MIME Content Type for BinHex Encoded Files
RFC1653 SMTP Service Extension for Message Size Declaration
RFC1652 SMTP Service Extension for 8bit-MIMEtransport
RFC1651 SMTP Service Extensions
RFC1641 Using Unicode with MIME
RFC1590 Media Type Registration Procedure
RFC1566 Mail Monitoring MIB
RFC1557 Korean Character Encoding for Internet Messages
RFC1556 Handling of Bi-directional Texts in MIME
RFC1555 Hebrew Character Encoding for Internet Messages
RFC1554 ISO-2022-JP-2: Multilingual Extension of ISO-2022-JP
RFC1544 The Content-MD5 Header Field
RFC1523 The text/enriched MIME Content-type
RFC1521 MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for
RFC1496 Rules for downgrading messages from X.400/88 to X.400/84 when MIME
RFC1489 Registration of a Cyrillic Character Set
RFC1468 Japanese Character Encoding for Internet Messages
RFC1456 Conventions for Encoding the Vietnamese Language VISCII: Vietnamese
RFC1437 The Extension of MIME Content-Types to a New Medium
RFC1428 Transition of Internet Mail from Just-Send-8 to 8Bit-SMTP/MIME
RFC1427 SMTP Service Extension for Message Size Declaration
RFC1426 SMTP Service Extension for 8bit-MIMEtransport
RFC1425 SMTP Service Extensions
RFC1212 Concise MIB Definitions
RFC1154 Encoding Header Field for Internet Messages
RFC1157 SNMP
RFC1123 Requirements for Internet Hosts - Communication Layers
RFC1090 SMTP on X.25
RFC1047 Duplicate messages and SMTP
RFC974 Mail Routing and the Domain System
RFC822 Standard for the Format of ARPA Internet Text Messages
RFC821 Simple Mail Transfer Protocol

 

About GSS

Global System Services Corporation (GSS) is the leading provider of consulting and professional services for large-scale and distributed infrastructure systems such as email and messaging, directory services, groupware, and wireless solutions. GSS customers include Fortune 500 companies, large services providers and telecom companies, government agencies, major messaging product vendors, and innovative technology startups.

GSS provides a complementary suite of services including strategic technology consultation and competitive vendor and product analysis, product and system architecture and design, system development deployment, customization, and testing, technical support, email migration, and other IT services. GSS has been directly responsible for some of the largest global systems and solutions and counts as customers many of the largest companies in the world.

From its offices in the Silicon Valley California, GSS delivers services and solutions to customers worldwide through a network of mobile consultants and qualified GSS Affiliates. With industry certified professionals on staff, GSS is a Qualified Lotus Business Partner, a Certified Microsoft Solution Provider (MCSP), a Principal Partner in the Sun Partner Advantage program and a member of the Sun Software Partner Council, as well as a member of key industry organizations.



Contact GSS

Global System Services Corporation (GSS)
650 Castro Street, Suite 120-268
Mountain View, CA 94041, U.S.A.
1 (650) 965-8669 phone
1 (650) 965-8679 fax
http://www.gssnet.com
info@gssnet.com


 
Messaging, Directory Services, Groupware


©1995-2005 by Global System Services Corporation (GSS). Portions of this material are copyright ©1995-1999 by Ron Herardian