0

Uploader needs file partials support

Currently there doesn't appear to be a way to ask Uploader to send up part of a file. This could be really useful for more sophisticated upload tools built on top of BrowserPlus that try to send only parts of files to a an upload service which is able to later piece these together to a complete file. The upload becomes more robust because the client is able to fail during a large upload, but only resume the part of the upload that might have failed.

So, in essence, the feature request would be for the Uploader service to take not just an map of files, but a map of file configuration maps. That is, the post var name points to a map (rather than a file handle) which specifies and a file offset, a size in bytes, and file handle for upload. (similar to FileAccess) This is an example of how the interface might change. In reality, I'm open to any design which allows the behavior above to be implemented easily (that is, partial file upload).

9 Replies
  • I am all for it. It would be a great feature and would allow better uploads, possibly smarter handling of large files. Any chance of getting this implemented?

    I was looking at Browser Plus for alternative "upload-in-chunks" possibilities, but it looks like right now it is not possible. Another idea that came to me was to make FileAccess read (parts of) binary files and allow those to be uploaded.

    Any other ideas for making this work?
    0
  • QUOTE (Michal Frackowiak @ Feb 20 2009, 02:00 AM) <{POST_SNAPBACK}>
    I am all for it. It would be a great feature and would allow better uploads, possibly smarter handling of large files. Any chance of getting this implemented?

    I was looking at Browser Plus for alternative "upload-in-chunks" possibilities, but it looks like right now it is not possible. Another idea that came to me was to make FileAccess read (parts of) binary files and allow those to be uploaded.

    Any other ideas for making this work?


    I find your suggestion elegant (fileaccess). The api could be fairly simple, you specify the chunk begining and size in bytes, and you get back another file handle.

    this can and will get implemented. we're expecting to get a pretty exciting new release out shortly, and after that our going to turn our focus back to services, most of em developed in the open :Dlloyd
    0
  • QUOTE (Lloyd Hilaiel @ Feb 28 2009, 04:27 PM) <{POST_SNAPBACK}>
    I find your suggestion elegant (fileaccess). The api could be fairly simple, you specify the chunk begining and size in bytes, and you get back another file handle.

    this can and will get implemented. we're expecting to get a pretty exciting new release out shortly, and after that our going to turn our focus back to services, most of em developed in the open :DWhat you really need from BP here is uploading in chunks. The above case could also allow pause + resume of uploads, handling failed chunk upload etc.
    0
  • QUOTE (Michal Frackowiak @ Mar 6 2009, 03:07 AM) <{POST_SNAPBACK}>
    I am glad to hear that. In fact we are just starting playing with BP, so I cannot (yet) suggest any precise API for it because I am still learning. I guess one of the approaches would be to create (virtual) file handlers similar to a way that you upload altered images. Those could point to a specific file portion (with offset and length), and you could easily use them with the Uploader.

    Another thing is Uploader - server communication, because here a semi-protocol should be designed, but I guess you could leave it for developers. It should be really easy once you can access files in chunks. What could happen for a single (large) file is:

    - Client initializes the upload, it uses a POST to a server with just some meta-data about a file (size, how many chunks)
    - Server provides unique URL for this upload
    - Client uploads chunks, one by one.
    - Client can ask Server about the status - e.g. which chunks are missing

    What you really need from BP here is uploading in chunks. The above case could also allow pause + resume of uploads, handling failed chunk upload etc.


    yay. so now that 2.2.0 is out I get to make promises. here's my promise. I'll get FileAccess on github by the end of next week. and start hacking in chunking support out in the open. all help is welcome.

    Next. As far as upload we've long envisioned at least 2 services. the first one, "Uploader" was designed to be a client side drop in replacement for existing forms. so that people wanting to integrate wouldn't even have to change their server side logic that recieves the upload.

    The *next* version would be something deep and custom, and more efficient. That thing could indeed be an exploration into new ideas and perhaps standards around file upload.

    The Uploader service should be on github in a month or so, and at that point we can start implementing something that is truely targeted at resumable and efficient uploads.

    Any pre-existing work we can leverage here?

    best,
    lloyd
    0
  • Just to follow up, there is support for partial file transfer now.

    Demo: http://browserplus.org/demos/resumable_uploads.html

    Service that makes this possible is FileAccess. Use either read() or slice().
    0
  • Hi Guys,

    Sorry to hijack this thread, but I've been having some problems using this and I've had no replies to my post in the Support & General Questions forum. I'm guessing there's not many people using this yet! ;)Matt
    0
  • Matt,

    I can't find your original post. Can you link to it here?

    robyn
    0
  • Already did. Sorry - probably wasn't clear enough. :urobyn
    0
  • Matt,

    I added a response in the original "Support & General Questions" forum.

    --Steve
    0
  • This forum is locked.

    Recent Posts

    in Feature Ideas