WebDAV upstreaming progress

Upstreaming in Radio: Upstreaming driver architecture.

Beta fileSystem Upstream Driver. Mmm, a template.

#upstream.xml is the equivalent of the ftpSite table. Need to find a way to map this to the WebDAV tool's specifications.

The WebDAV RFC is RFC 2518.

Useful places in the root:

radio.upstream.builtinDrivers is where UserLand's drivers go.

Added drivers go in user.radio.drivers.upstream.

Format for various #upstream.xml files:

• xmlStorageSystem:

  	<upstream type="xmlStorageSystem" version="1.0">
  		<server>radio.xmlstoragesystem.com</server>
  		<port>80</port>
  		<usernum>100148</usernum>
  		<passwordName>default</passwordName>
  		<name>Nicholas Riley</name>
  		<protocol>xml-rpc</protocol>
  		<rpcPath>/RPC2</rpcPath>
  		<soapAction>/xmlStorageSystem</soapAction>
  	</upstream>
  	

• ftp:

  	<upstream type="ftp" version="1.0">
  		<username>nicholas</username>
  		<passwordName>nameOfPassword</passwordName>
  		<server>ftp.somesite.dom</server>
  		<path>/path/to/upload/folder</path>
  		<url>http://base.url.for.site.dom/something/</url>
  	</upstream>
  	

• fileSystem:

  	<upstream type="fileSystem" version="1.0">
  		<folder>GrayExtra:Sample Upstream Folder:</folder>
  		<url>http://base.url.for.site.dom/something/</url>
  		<server>Name of this server</server>
  	</upstream>
  	

The format for a WebDAV #ftpSite (XMLized, it's really just a table):

<account>nicholas</account>
<directory>/path/to/webdav/upload/folder</directory>
<domain>webdav.server.dom</domain>
<password>thePassword</password>
<port>8080</port>
<url>http://base.url.for.site.dom/something/</url>

The parts that are actually used: domain, port (optional, defaults to 80, you can also include a colon after the domain), account, password.

Another possibility is to use a URL for specifying the WebDAV server. I thought of this the first time around, too. But this presents the question: where do you stop with the URL? Not everyone knows the right URL format (http://username:password@site) or all the right escaping conventions. And to be a good Radio citizen, the password shouldn't be strewn in the filesystem anyway. So I think I'll leave it as it is. If it becomes necessary to support other formats, for example https (looking forward to testing that TLS extension when it comes out!) then I'll add another field, say 'protocol' (whoops, already used) or 'transport'.

Here's a hypothetical, corresponding WebDAV #upstream.xml, then:

<upstream type="webdav" version="1.0">
	<server>webdav.server.dom</server>
	<port>8080</port>
	<username>nicholas</username>
	<passwordName>webdavPasswordName</passwordName>
	<url>http://base.url.for.site.dom/something/</url>
	<path>/path/to/webdav/upload/folder</path>
</upstream>

Hm, note the overloading of 'server' above, it's used for a domain and a non-domain entity in two cases. I don't like it. Oh well.

I also don't like the way passwordName-to-password mapping is done with a table lookup rather than a function. This seems like an ideal candidate to be abstracted, so, for example, the Mac keychain can be used as a back end instead of insecure storage in the Radio root. (Although there are some UI interactivity issues associated with use of the keychain which I'll ignore for the moment...)

WebDAV filewriter shutdown: site root begins with / - fix

More stuff... need to implement digest authentication and the 'auth' option. Some info on this came from the Mac OS X 10.1.1 to 10.1.3 technote: 10.1.1 added digest authentication and 10.1.3 made further enhancements.

The referenced RFCs:

• RFC 2617: digest authentication
• RFC 2616: HTTP 1.1

I've duplicated tcp.httpClient (which I've already fixed some bugs in) for my own use... need to ask UserLand about the legality of distributing modified versions of their code, but for the time being I'm just hacking on it.

Python's support for 'extended HTTP functionality and WebDAV' is covered by PEP 268.

A sample, incomplete implementation of PEP 268 is checked into the Python CVS here