To know about HTML5
This tutorial provides a guide and code examples for leveraging the large file upload using File API inside of a Web Worker. While we are trying to upload large file, we getting the problems like browser showing kill pages or unresponsive page etc.. This are because of we are blocking the user agent and we are utilising lot of CPU, it will lead to performance down or user agent getting continuos processing. We need to avoid the blocking or performance utilization on document. Solution for that is simple, we need to upload file using background processing nothing but using webworkers. By using webworkers we can upload files in background so that no errors or problems will occur.
In some cases reading the entire file into memory isn't the best option. For example, say you wanted to write an async file uploader. One possible way to speed up the upload would be to read and send the file in separate byte range chunks. The server component would then be responsible for reconstructing the file content in the correct order. Lucky for us, the File interface supports a slice method to support this use case. The method takes a starting byte as its first argument, ending byte as its second, and an option content type string as a third.
I was demonstrated the upload using file input and drag and drop. you could use client-side logic to verify an upload's mimetype matches its file extension or restrict the size of an upload. The most straightforward way to load a file is to use a standard <input type="file"> element. JavaScript returns the list of selected File objects as a FileList. Here's an example that uses the 'multiple' attribute to allow selecting several files at once. Another technique for loading files is native drag and drop from the desktop to the browser. This example include drag and drop support. The following example demonstrates reading chunks of a file.
If you load models or textures from external files, due to browsers' "same origin policy" security restrictions, loading from a file system will fail with a security exception. To solve this Start Chrome executable with a command line flag: chrome --allow-file-access-from-files or else run from local server
This will solve the following problems:
- How to upload large file more than 1GB
- Synchronous File Uploading
- Large File upload with XHR2 uisng webworkers
- Drag and Drop File upload
- Multiple File Upload without blocking UI
- Increasing performance While File uploads
- Background Processing of file uploads
- Slicing the file
HTML
<input type="file" id="files" name="files[]" multiple /> <div id="drop_zone"> Drop files here </div> <output id="list"></output>
JavaScript
var worker = new Worker('fileupload.js'); worker.onmessage = function(e) { alert(e.data); } worker.onerror =werror; function werror(e) { console.log('ERROR: Line ', e.lineno, ' in ', e.filename, ': ', e.message); } function handleFileSelect(evt) { evt.stopPropagation(); evt.preventDefault(); var files = evt.dataTransfer.files||evt.target.files; // FileList object. worker.postMessage({ 'files' : files }); //Sending File list to worker // files is a FileList of File objects. List some properties. var output = []; for (var i = 0, f; f = files[i]; i++) { output.push('<li><strong>', escape(f.name), '</strong> (', f.type || 'n/a', ') - ', f.size, ' bytes, last modified: ', f.lastModifiedDate ? f.lastModifiedDate.toLocaleDateString() : 'n/a', '</li>'); } document.getElementById('list').innerHTML = '<ul>' + output.join('') + '</ul>'; } function handleDragOver(evt) { evt.stopPropagation(); evt.preventDefault(); evt.dataTransfer.dropEffect = 'copy'; // Explicitly show this is a copy. } // Setup the dnd listeners. var dropZone = document.getElementById('drop_zone'); dropZone.addEventListener('dragover', handleDragOver, false); dropZone.addEventListener('drop', handleFileSelect, false); document.getElementById('files').addEventListener('change', handleFileSelect, false);
Worker (fileupload.js) :
var file = [], p = true;
function upload(blobOrFile) {
var xhr = new XMLHttpRequest();
xhr.open('POST', '/server', false);
xhr.onload = function(e) {
};
xhr.send(blobOrFile);
}
function process() {
for (var j = 0; j <file.length; j++) {
var blob = file[j];
const BYTES_PER_CHUNK = 1024 * 1024;
// 1MB chunk sizes.
const SIZE = blob.size;
var start = 0;
var end = BYTES_PER_CHUNK;
while (start < SIZE) {
if ('mozSlice' in blob) {
var chunk = blob.mozSlice(start, end);
} else {
var chunk = blob.webkitSlice(start, end);
}
upload(chunk);
start = end;
end = start + BYTES_PER_CHUNK;
}
p = ( j = file.length - 1) ? true : false;
self.postMessage(blob.name + " Uploaded Succesfully");
}
}
self.onmessage = function(e) {
for (var j = 0; j < e.data.length; j++)
files.push(e.data[j]);
if (p) {
process()
}
}
Conclusion
I hope this article has help you to upload large files. Web Workers are an underutilized and under-appreciated feature of HTML5. We can minimize the work to upload a large file. The technique is to slice the upload into multiple chunks, spawn an XHR for each portion, and put the file together on the server. This is similar to how GMail uploads large attachments so quickly.
Further Reading
35 comments
commentsHi,
ReplyThank you for this example.
Could you share the server script as well? I don't really get how to save the file server-side.
Thank you in advance,
Valerie
While this is a great overview, I wasn't able to implement due to a few bugs.
ReplyBro...change globally theme..it's hard read using mobile browsers
ReplyBro...change globally theme..it's hard read using mobile browsers
Replyhi lets look at the post.
ReplyI will share php script in a quick session
Replyconsloe.log - in first javascript - must be console.log,
Replyexpect that little thing - Great Post
thanks
post me bugs I can resolve.
ReplyUpdates: webkitslice prefix removed so you can use as it as in specification.
ReplyHi. Thanks for sharing.
ReplyDunno if I miss something but when I select file, I get this error in console "DataCloneError: The object could not be cloned.
'files' : files"
I search on the web but don't really find what can be the reason of this error.
As previously ask, could you share a server-side script (PHP ?) to rebuilt file on the server.
Thank you in advance.
Great article.
ReplyCan you provide the php script to try this on my computer and test how much memory will consume?
Thanks in advance
Great Article,
ReplyHi can you please send the php script.
Great! Could you please share the php script? Thanks in advance, Jan
ReplyIt would be nice if you could attach the working project for reference.
Replyokay, I would Place working project ASAP.
ReplyCan you please share how to save the file on the server side?
Replyplease share how to save the file on server side?
Replyplease share how to save this file on server particular location i.e. shared location on server?
ReplyThe blob,webkitSlice or blob.WebSlice is not working in safari could you please suggest me any alternative.Tried few snippets nothing worked the method should return a blob object which has size and type of blob. Any help or pointer appreciated
Replyhi Rakesh,
ReplywebkitSlice deprecated use blob.slice
i am also interested in seeing your serverSide script. was that posted @ some point? thank you for the share
ReplySuch a great post! Any idea for the server side code using ASP.NET
Replyfew days ago I was browsing and found a file upload site quickly, besides that we receive payments from the file that we upload it if there is downloading perfectly. file2up.net
ReplyHi. Can you post your server side code? I am having a difficulty on that side. Cause I can already transfer the data of the file to the server, but i cant transfer the file itself to the server. Im confused on how should i do it. I hope you can help me about this. Thankyou.
ReplyNice. Please attach sample project
ReplyHi, thanks for your great work.
Replyif you can, can you upload server side php script please
Scenario: Your worker script is running in the background and all is fine and the user is able to continue using the page while the worker is uploading. All of a sudden, the user decides to click on some link that happens to be on the page, and the page navigates away from the site. The worker is no longer accessible and dependent on the browser it may or may quit right away, but the fact is, the upload will be interrupted before it is finished.
ReplyQuestion: How would you go about detecting any navigation away from the page that would kill the worker, and presenting the user with a warning about it? Have you run into that scenario before and what was your solution if you did?
Such a great post! Any idea for the server side code using ASP.NET
Replyhref="https://sisgain.com/webrtc"webrtc development
(failed)net::ERR_CONNECTION_RESET
ReplyIn server side you have to collect the all the chunks and then use file stream to create file using content-type.
Reply"Nicely explained. To support the WebRTC app development, I would like to share that In terms of global coverage, the WebRTC market spans North America, Europe, Asia, the Middle East, South America, and Africa. Nowadays, Google puts great efforts into the development of Web Real-Time Communication. The top WebRTC development companies are growing rapidly with the increasing demand of the technology.
ReplyVisit zyngler.com! Visit Play Free Online Games For Kids!
ReplyFor the PHP script
ReplyJust do file_get_contents("php://input");
That gets you the data.. from there,
You wanna use fopen, and fwrite to save it to a file and whatever the extension is.. PNG, mp4 etc.. but fuck this website and fuck this dude..
Raju konga can suck dick
Replyi jst wanted to be here
Reply