| RFC |
| TOC |
| Network Working Group | B. de hÓra |
| Request for Comments: | Propylon Ltd. |
| <draft-httplr-01.txt> | February 2005 |
| Category: Informational |
HTTPLR
draft-httplr-01.txt
This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.
Copyright (C) The Internet Society (2005). All Rights Reserved.
This document describes HTTPLR revision 01, an application protocol for reliable transmission of messages using HTTP. The protocol provides a measure of reliability within the client server model of HTTP without recourse to a peer to peer model, where both communicators are HTTP servers.
This draft HAS NOT been submitted for publication, and does not have any status; it should be referred to as a "pre-draft."
Please submit comments to the draft author.
Sections marked out like this, @@@world domination@@@, indicate editorial notes that MUST be removed prior to publication.
| RFC |
| TOC |
1
Requirements notation
2
Introduction
3
Requirements and Assumptions
3.1
Requirements
3.1.1
Agreement
3.1.2
Message Duplication
3.1.3
Message Opacity
3.1.4
URI Opacity
3.1.5
Low impact on the Client
3.1.6
Fidelity with HTTP
3.2
Assumptions
3.2.1
Eventual Arrival
3.2.2
Regression
4
URI state resources
5
HTTP Methods and message delivery
5.1
Identification of exchanges
5.2
Identifier generation
6
Retries and timeouts
7
State management and duration
8
The Upload Protocol
8.1
Step One: Establish exchange URL
8.1.1
Phantom Exchanges
8.2
Step two: Send message
8.2.1
Behaviour where PUT delivery is not supported
8.2.2
Behaviour where delivery is repeated
8.3
Step three: Message Reconciliation
8.3.1
Behaviour where DELETE reconciliation is not supported
8.3.2
Rejection of out of order DELETE requests
9
The Download Protocol
9.1
Step One: Establish feed URL
9.1.1
Constraints on the Atom feed format
9.2
Step Two: Download Message
9.3
Step Three: Message Reconciliation
9.3.1
Rejection of out of order requests
10
Security Considerations
11
Appendix A: HTTP Response Codes
12
Revision History
13
References
Author's Address
Intellectual Property and Copyright Statements
Figure 1: Creating an exchange URL
Figure 2: Indicating the supported methods for an exchange URL
Figure 3: Sending a message to the exchange URL
Figure 4: Indicating the supported methods for an exchange URL on rejecting PUT
Figure 5: Refusing to accept an already accepted message
Figure 6: Reconciling a sent message
Figure 7: Indication the message is gone
Figure 8: Informing a client the message has been reconciled
Figure 9: Indicating the supported methods for an exchange URL on rejecting PUT
Figure 10: Refusing to end an exchange before accepting a message
Figure 11: Obtaining a list of available messages
Figure 12: Downloading a message
Figure 13: Indicating acceptance that a message has already been downloaded
Figure 14: Reconciling a downloaded message
Figure 15: Indicating a downloaded message is gone
Figure 16: Refusing to complete an exchange before the message is downloaded
| TOC |
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
A "Client" in this discussion is whomever begins the reliable exchange via a HTTP request and a "Server" is the origin server responding to the request. A "Message" is the entity body sent by a request to a Server from a Client. The "Network" encompasses all things that lie between a Server and a Client, such as caches, proxies, ethernet cables, Internet backbones and so on.
| TOC |
This document describes an application protocol for guaranteed once and only once transmission of messages using HTTP, something that HTTP alone does not guarantee. It describes a means for both downloading and uploading of messages. It is not concerned with endpoint availability, robustness of components, or details of persistent storage. It is not concerned with message order.
A characteristic of distributed systems is that senders and receivers of messages can't know with certainty what went wrong in the event of failure, and without catering for agreement, they might not know if anything did go wrong with a transmission. Our primary concern for failure is dealing with partial failure. Partial failure is where one component in the system fails while the others continue to function. The HTTP client-server model has three failing parts, the Client, the Network, and the Server. For example, if the Network fails mid-transmission, a request might be arrive to the Server but not a response to the Client. Or if the Server's firewall rules are mis-configured, Client requests might be rejected out of hand.
The techniques described here provides a measure of reliability within the client server model of HTTP. Reliable variants of HTTP or protocols layered upon HTTP often require a peer to peer model, where both communicators are HTTP servers.
The first published description of a reliable protocol using a HTTP client and server is attributed to Paul Prescod [Prescod]. That document 'Reliable delivery in HTTP', along with the author's experiences implementing messaging systems with HTTP and anecdotal descriptions of ad-hoc applications created to handle reliability for HTTP were the basis for this protocol.
| TOC |
The key requirement for the protocol is that the Client and Server can reach agreement on the success of a message exchange.
The protocol MUST NOT result in duplicated messages. Parties are required to hold to state during the exchange by using a URL as a shared key, until they come to an agreement that the message has indeed been delivered.
The protocol MUST NOT rely on the contents of a Message to complete an exchange.
The protocol MUST NOT rely on the contents of a URI to complete an exchange.
The Client MUST NOT need to run a web server to complete the reliable contract or expose a port to the outside world.
The protocol MUST NOT be compromised by caching proxies or firewall rules. The protocol MUST NOT contravene existing HTTP specifications. Therefor it MUST be consistent with HTTP semantics rather than extending or abusing them.
The protocol makes an important assumption - an infinite number of requests will result in an infinite number of responses. This assumption is known as 'eventual arrival'. It allows us to disregard arbitrary (often called 'Byzantine') failure modes for which a reliable protocol can never be modelled.
Theoretically, as HTTP is an asymmetric protocol, we keep needing the Client to send a request to the Server acknowledging receipt of the last response, so we regress infinitely. Given the eventual arrival assumption, we can ignore such regression.
| TOC |
HTTPLR manages message exchanges through state transitions associated with a URI called the 'exchange URL'. The following URIs denote exchange states:
The URIs are used as normative state identifiers in this memo's prose. Client and Server implementations MAY use these URIs to denote or share state information (for example as RDF statements or within an XML document) between themselves or other interested parties. Implementers MUST treat the trailing '/' on the URIs as significant.
| TOC |
One way to make sure a message went through in HTTP is to resend until the Client gets an appropriate response from the Server. Some HTTP actions (GET, PUT, DELETE) enable this in principle through idempotence - repeating the action does not alter the result of the first successful action. The HTTP POST method which is popular for use in message transmission is not idempotent; a message sending strategy based purely on Client retries using POST is not guaranteed to be safe.
One way to avoid overwriting data with a sequence of HTTP methods is through the use of a identifier in a header or the message body. As HTTP is a stateless protocol, it does not require the use of message identifiers; use of identifiers thus requires additional mandatory constraints on the Client and Server to maintain state. Assigning such identifiers to message exchanges is a known networking idiom (for example it is used in TCP). [Lynch] formally describes the general process of using a shared identifier to reach agreement in the "FivePacketHandshakeProtocol".
It is necessary for either the Client or the Server to generate a shared identifier. It SHOULD be noted by implementers that the algorithms for generating highly unique identifiers (like GUIDs) are complex and can be difficult to get right. A published algorithm and implementation for GUIDs is described in [Leach]. As well as being unique, identifiers should be stable across time. As a result of the need for uniqueness and stability, this task is entrusted to the Server through its assignation of URLs which are communicated using the Location header. Using only one generation source reduces the likelihood of error, particularly where there are multiple Clients.
| TOC |
Under failures in requests and responses, the onus to continue the exchange is placed on the Client. The number of times a Client retries to send a request in order to get back a Server response and the duration between retries is not defined here.
| TOC |
Once an exchange begins, the Server and Client MUST hold exchange state until one the following happens :
| TOC |
This section specifies the basic protocol for delivery of messages to a Server from a Client.
Message exchange state is coordinated through a shared resource called the "exchange URL". This resource is distinct from the exchanged message. The exchange URL MUST be unique. This implies that an exchange URL SHALL NOT be recycled.
A request-response exchange between the Client and Server at a well-known URL establishes the exchange URL. How the Client and Server determine the well-known URL is not specified here. The Client MUST initiate the exchange using POST. If the Server is willing to accept the exchange request it MUST use the 201 Created response code. The identifier supplied by the Server MUST be a URL, which MUST appear in the Location header of the response. This is the exchange URL. Here is an example:
C: POST www.example.org/rmservice HTTP/1.1 S: HTTP/1.1 201 Created S: Location:www.example.org/rmservice?id=249D6557
Figure 1: Creating an exchange URL
The server MAY return a representation (entity body) in the response.
In the event that an opening request fails, the Client MAY repeatedly request an exchange URL until it receives a response.
After receiving the request and prior to sending the response, the Server MUST maintain state about the exchange URL accordingly:
After receiving the response, the Client MUST maintain state about this URL accordingly:
If for some reason, the Server cannot durably record the state, it MUST inform the client by returning a 500 Internal Error status code and the response MUST NOT contain a Location header. 'Durably record' is understood to imply persistent storage outside working memory (for example state can be expected to be maintained between Server reboots).
If the Client requests an exchange URL, but did not receive the response from the Server, the Server may be holding onto an exchange URL to which the client will never send a Message (the client will simply ask for another URL). These are "Phantom Exchanges".
As the existence of Phantom Exchanges is not actively harmful, a means for removing them is not specified. However, If the Client was allowed to examine a list of incomplete exchanges, it could identify Phantom Exchanges and terminate them. Following feedback from implementations, future HTTPLR revisions MAY provide an exchange pattern as an optimization to allow the Server to release resources associated with such exchange URLs.
The Client MAY use one of PUT or POST to send its Message to the exchange URL supplied by the Server. The Client MAY use a HEAD request to determine what methods are supported at any point in the exchange. The Server MUST support HEAD requests against exchange URLs at any point in the exchange:
C: HEAD www.example.org/rmservice?id=249D6557 HTTP/1.1 S: HTTP/1.1 200 Ok S: Allow: GET, HEAD, POST
Figure 2: Indicating the supported methods for an exchange URL
Given the option, the Client SHOULD prefer the PUT over POST. The Server MUST support the POST delivery option and SHOULD support PUT.
The Client request MUST contain an entity body (the Message). The Server response SHOULD include a Location header naming the exchange URL in its response. The Server response MUST include an Allow header indicating which methods the Client MAY use to continue the exchange. Here is an example:
C: PUT www.example.org/rmservice?id=249D6557 HTTP/1.1 C: [crlf] C: [message body] S: HTTP/1.1 202 Accepted S: Location:www.example.org/rmservice?id=249D6557 S: Allow: GET, HEAD, POST, DELETE
Figure 3: Sending a message to the exchange URL
After receiving the request and prior to sending the response, the Server MUST maintain state about this exchange URL accordingly:
After receiving the response, the Client MUST maintain state about this URL accordingly:
The Client MUST not send Messages to a URL it has recorded as purl.oclc.org/httplr/state/accepted/.
If the Server does not support PUT to an exchange URL and PUT is requested, it MUST return a 501 Not Implemented status code and it MUST indicate the allowed methods at the URL.
C: PUT www.example.org/rmservice?id=249D6557 HTTP/1.1 C: [crlf] C: [message body] S: HTTP/1.1 501 Not Implemented S: Allow: GET, HEAD, POST
Figure 4: Indicating the supported methods for an exchange URL on rejecting PUT
In the event that the Server is not responding as expected or there is a Network failure (such as a timeout), the Client MAY try to resend the Message in the absence of a response. In this case it is possible that the Server has already recorded the message as being in the purl.oclc.org/httplr/state/accepted/ state but that the response had not arrived at the Client. To avoid duplicate delivery, the Server MUST respond to further POST or PUT requests with entity bodies against an exchange URL recorded as being in the purl.oclc.org/httplr/state/accepted/ state with a 405 Method Not Allowed response. The Server SHALL NOT attempt to verify that the Message sent is the same as before. The Server SHOULD include the exchange URL in a Location header in its responses. For example:
C: PUT www.example.org/rmservice?id=249D6557 HTTP/1.1 C: [crlf] C: [message body] S: HTTP/1.1 405 Method Not Allowed S: Location:www.example.org/rmservice?id=249D6557 S: Allow: GET, HEAD
Figure 5: Refusing to accept an already accepted message
The Client on seeing a 405 response code for a Message exchange in the purl.oclc.org/httplr/state/created/ state SHOULD assume the Message has been previously transmitted and SHOULD record the state as purl.oclc.org/httplr/state/accepted/.
Up to this point to Server knows the Message was sent to it, but does not know if the Client agrees it has been sent (as it does not know for certain if the Client received the response). The Client MUST inform the Server with a DELETE or POST request that it is in agreement the message was delivered - note that the Server will have indicated what methods are supported. Given the option, the Client SHOULD prefer DELETE:
C: DELETE www.example.org/rmservice?id=249D6557 HTTP/1.1 S: HTTP/1.1 200 Ok S: Location: www.example.org/rmservice?id=249D6557
Figure 6: Reconciling a sent message
The Client reconciliation request MUST NOT contain an entity body. In the case where the POST method is used for both the message delivery and the reconciliation request, the Server MUST use the absence of an entity to distinguish the request order. When the Client receives the Server response, it MAY release recorded state about the exchange URI.
The Server response SHOULD contain a Location header to indicate the exchange URL. On receiving a reconciliation request the Server MUST respond to further such requests with a 410 Gone response code. For example:
C: DELETE www.example.org/rmservice?id=249D6557 HTTP/1.1 S: HTTP/1.1 410 Gone S: Location: www.example.org/rmservice?id=249D6557 S: Allow: GET, HEAD
Figure 7: Indication the message is gone
After receiving the request and prior to sending the response, the Server MUST maintain state about this exchange URL accordingly:
After receiving the response with the message, the Client MUST maintain state about this URL accordingly:
In the event that the Server is not responding as expected or there is a Network failure (such as a timeout), the Client MAY try to resend the reconciliation in the absence of a response. In this case it is possible that the Server has already recorded the message as being in the purl.oclc.org/httplr/state/finished/ state but that the response had not arrived at the Client. The Server MUST respond to further POST or PUT request against an exchange URL recorded as being in the purl.oclc.org/httplr/state/finished/ state with a 410 Gone response. The Server MUST include an Allow header in its response. The Server SHOULD include the exchange URL in a Location header in its responses. For example:
C: DELETE www.example.org/rmservice?id=249D6557 HTTP/1.1 C: [crlf] S: HTTP/1.1 410 Gone S: Location:www.example.org/rmservice?id=249D6557 S: Allow: GET, HEAD
Figure 8: Informing a client the message has been reconciled
The Client on seeing a 410 response code for a reconciliation request in the purl.oclc.org/httplr/state/accepted/ state SHOULD agree the Message was exchanged and SHOULD NOT send further reconciliation requests to the Server.
If the Server does not support DELETE to an exchange URL to reconcile and DELETE is requested, it MUST return a 501 Not Implemented status code and it MUST indicate the allowed methods at the URL.
C: DELETE www.example.org/rmservice?id=249D6557 HTTP/1.1 C: [crlf] S: HTTP/1.1 501 Not Implemented S: Allow: GET, HEAD, POST
Figure 9: Indicating the supported methods for an exchange URL on rejecting PUT
A Client MAY send a DELETE request to an exchange URL it has recorded as being in the purl.oclc.org/httplr/state/accepted/ state. To ensure safety against out of order requests, the Server MUST respond to DELETE requests against an exchange URL it has recorded in the purl.oclc.org/httplr/state/created/ state with a 405 Not Allowed response. For example:
C: POST www.example.org/rmservice?id=249D6557 HTTP/1.1 S: HTTP/1.1 405 Not Allowed S: Location: www.example.org/rmservice?id=249D6557 S: Allow: GET, HEAD, POST
Figure 10: Refusing to end an exchange before accepting a message
| TOC |
The Client when downloading messages needs to determine either the URL of a message to download or a list of such URLs.These URLs are called 'Message URLs'. They are to be found at another URL called the "feed URL". How the Client discovers a Server's feed URL is not specified here.
To fetch the list of available messages, the Server MUST use a GET request against the feed URL. The Server MUST at minimum support the Atom syndication format to publish available messages and MUST indicate that by setting the response Content-Type header to application/x.atom+xml. The Server MAY present other content formats. The Client MUST be able to process the Atom syndication format at minimum [ATOM]. Here's an example:
C: GET www.example.org/e4/c234 HTTP/1.1 S: HTTP/1.1 200 Ok S: Content-Type: application/x.atom+xml S: [crlf] S: <atom:feed version="draft-ietf-atompub-format-05: do not deploy" S: xmlns:atom="purl.org/atom/ns#draft-ietf-atompub-format-05"> S: <atom:head> S: <atom:title>Example Feed</atom:title> S: <atom:link class="www.example.org/e4/c234/"/> S: <atom:updated>2004-12-17T00:00:00Z</atom:updated> S: <atom:author> S: <atom:name>Lemmy</atom:name> S: </atom:author> S: </atom:head> S: <atom:entry> S: <atom:title></atom:title> S: <atom:link class="www.example.org/e4/c234/item/01"/> S: <atom:id>www.example.org/e4/c234/item/01</atom:id> S: <atom:updated>2004-12-17T00:00:00Z</atom:updated> S: </atom:entry> S: </atom:feed>
Figure 11: Obtaining a list of available messages
The Client MUST use GET to collect a Message from the Message URL supplied by the Server. The Server response MUST contain a Location header to indicate the exchange URL. The Message URL from which the Message representation is collected MUST be distinct from the exchange URL that will be used by the client to acknowledge message receipt.
C: GET www.example.org/e4/c234/item/01 HTTP/1.1 S: HTTP/1.1 200 Ok S: Location:www.example.org/e4/gipoco.com is neither affiliated with the authors of this page or responsible
for its contents. This is a safe-cache copy of the original web site.gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.